2024-09-19T07:10:33.504 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T07:10:33.690 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T07:10:33.776 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911794 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/iozone}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911794' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi159.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGER7H/4Aoy1yZSRSVLdxtug1pKQrA14jb/yY3JFK+UXEoJWy+Rh3uPHbZx41DlgqW9j6Hwn3CsjarMyWePRB+A= smithi189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMudL8I3HehFi2jjdQKufcEy5cL+U+MOCstTcASsiFXFqqfF3qH4NFkNkVHjKEVQ3VuidUq77yxxwTmZ5o5YRuQ= smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMs+aTwDMR0ms94qS2kdpqksr2QTqjcsDWmE4l3SD+a1AhDCrdfUWi7qVzAUYC6CQyr6m+dwc5TBr9WcH6mT0gY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T07:10:33.776 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T07:10:33.777 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T07:10:33.777 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T07:10:33.782 INFO:teuthology.task.internal:Checking packages... 2024-09-19T07:10:33.804 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T07:10:33.804 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T07:10:33.804 INFO:teuthology.packaging:ref: None 2024-09-19T07:10:33.804 INFO:teuthology.packaging:tag: None 2024-09-19T07:10:33.804 INFO:teuthology.packaging:branch: squid 2024-09-19T07:10:33.805 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:10:33.805 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T07:10:34.014 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T07:10:34.016 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T07:10:34.329 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T07:10:34.329 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T07:10:34.400 INFO:teuthology.task.internal:Saving configuration 2024-09-19T07:10:34.420 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T07:10:34.435 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T07:10:34.461 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911794', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 07:03:28.635389', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGER7H/4Aoy1yZSRSVLdxtug1pKQrA14jb/yY3JFK+UXEoJWy+Rh3uPHbZx41DlgqW9j6Hwn3CsjarMyWePRB+A='} 2024-09-19T07:10:34.483 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911794', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 07:03:28.633412', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMudL8I3HehFi2jjdQKufcEy5cL+U+MOCstTcASsiFXFqqfF3qH4NFkNkVHjKEVQ3VuidUq77yxxwTmZ5o5YRuQ='} 2024-09-19T07:10:34.505 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911794', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 07:03:28.636942', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMs+aTwDMR0ms94qS2kdpqksr2QTqjcsDWmE4l3SD+a1AhDCrdfUWi7qVzAUYC6CQyr6m+dwc5TBr9WcH6mT0gY='} 2024-09-19T07:10:34.505 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T07:10:34.513 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-19T07:10:34.513 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-19T07:10:34.513 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-19T07:10:34.513 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T07:10:34.621 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0362c5fbe0>, signals=[15]) 2024-09-19T07:10:34.621 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T07:10:34.626 INFO:teuthology.task.internal:Opening connections... 2024-09-19T07:10:34.626 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi159.front.sepia.ceph.com 2024-09-19T07:10:34.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:10:34.702 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi189.front.sepia.ceph.com 2024-09-19T07:10:34.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:10:34.777 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2024-09-19T07:10:34.777 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:10:34.852 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T07:10:34.860 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2024-09-19T07:10:34.879 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2024-09-19T07:10:34.879 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-09-19T07:10:34.937 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-09-19T07:10:34.937 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9" 2024-09-19T07:10:34.937 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-09-19T07:10:34.937 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-09-19T07:10:34.937 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9" 2024-09-19T07:10:34.937 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9" 2024-09-19T07:10:34.937 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T07:10:34.938 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-09-19T07:10:34.938 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon" 2024-09-19T07:10:34.938 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T07:10:34.938 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-09-19T07:10:34.938 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T07:10:34.938 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T07:10:34.938 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T07:10:34.939 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-09-19T07:10:34.961 DEBUG:teuthology.orchestra.run.smithi189:> uname -m 2024-09-19T07:10:34.980 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2024-09-19T07:10:34.980 DEBUG:teuthology.orchestra.run.smithi189:> cat /etc/os-release 2024-09-19T07:10:35.038 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="CentOS Stream" 2024-09-19T07:10:35.038 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="9" 2024-09-19T07:10:35.038 INFO:teuthology.orchestra.run.smithi189.stdout:ID="centos" 2024-09-19T07:10:35.039 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE="rhel fedora" 2024-09-19T07:10:35.039 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="9" 2024-09-19T07:10:35.039 INFO:teuthology.orchestra.run.smithi189.stdout:PLATFORM_ID="platform:el9" 2024-09-19T07:10:35.039 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T07:10:35.039 INFO:teuthology.orchestra.run.smithi189.stdout:ANSI_COLOR="0;31" 2024-09-19T07:10:35.039 INFO:teuthology.orchestra.run.smithi189.stdout:LOGO="fedora-logo-icon" 2024-09-19T07:10:35.039 INFO:teuthology.orchestra.run.smithi189.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T07:10:35.039 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://centos.org/" 2024-09-19T07:10:35.039 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T07:10:35.039 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T07:10:35.039 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T07:10:35.040 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2024-09-19T07:10:35.061 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2024-09-19T07:10:35.080 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2024-09-19T07:10:35.081 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2024-09-19T07:10:35.139 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="CentOS Stream" 2024-09-19T07:10:35.139 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="9" 2024-09-19T07:10:35.139 INFO:teuthology.orchestra.run.smithi203.stdout:ID="centos" 2024-09-19T07:10:35.139 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="rhel fedora" 2024-09-19T07:10:35.139 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="9" 2024-09-19T07:10:35.139 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el9" 2024-09-19T07:10:35.139 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T07:10:35.139 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31" 2024-09-19T07:10:35.139 INFO:teuthology.orchestra.run.smithi203.stdout:LOGO="fedora-logo-icon" 2024-09-19T07:10:35.140 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T07:10:35.140 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://centos.org/" 2024-09-19T07:10:35.140 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T07:10:35.140 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T07:10:35.140 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T07:10:35.140 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2024-09-19T07:10:35.170 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T07:10:35.180 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T07:10:35.195 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T07:10:35.195 DEBUG:teuthology.orchestra.run.smithi159:> test '!' -e /home/ubuntu/cephtest 2024-09-19T07:10:35.198 DEBUG:teuthology.orchestra.run.smithi189:> test '!' -e /home/ubuntu/cephtest 2024-09-19T07:10:35.200 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2024-09-19T07:10:35.217 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T07:10:35.224 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T07:10:35.224 DEBUG:teuthology.orchestra.run.smithi159:> test -z $(ls -A /var/lib/ceph) 2024-09-19T07:10:35.255 DEBUG:teuthology.orchestra.run.smithi189:> test -z $(ls -A /var/lib/ceph) 2024-09-19T07:10:35.258 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2024-09-19T07:10:35.362 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T07:10:35.437 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T07:10:35.451 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-19T07:10:35.452 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-19T07:10:35.452 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-19T07:10:35.452 DEBUG:teuthology.orchestra.run.smithi159:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T07:10:35.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:10:35.471 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-19T07:10:35.649 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-19T07:10:35.649 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-19T07:10:35.649 DEBUG:teuthology.orchestra.run.smithi159:> uname -r 2024-09-19T07:10:35.668 INFO:teuthology.orchestra.run.smithi159.stdout:5.14.0-503.el9.x86_64 2024-09-19T07:10:35.668 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-19T07:10:35.668 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-19T07:10:35.668 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum upgrade -y linux-firmware 2024-09-19T07:10:38.211 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.3 MB 00:00 2024-09-19T07:10:40.237 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-19T07:10:46.179 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.5 MB 00:02 2024-09-19T07:10:47.722 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 19 kB 00:00 2024-09-19T07:10:48.892 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-19T07:10:54.298 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-09-19T07:10:56.254 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-19T07:10:56.264 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-19T07:10:56.264 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2024-09-19T07:10:56.264 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-19T07:10:56.264 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrading: 2024-09-19T07:10:56.265 INFO:teuthology.orchestra.run.smithi159.stdout: linux-firmware noarch 20240905-146.el9 baseos 388 M 2024-09-19T07:10:56.265 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:10:56.265 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-19T07:10:56.265 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-09-19T07:10:56.265 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrade 1 Package 2024-09-19T07:10:56.265 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:10:56.265 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 388 M 2024-09-19T07:10:56.265 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-19T07:11:01.758 INFO:teuthology.orchestra.run.smithi159.stdout:linux-firmware-20240905-146.el9.noarch.rpm 72 MB/s | 388 MB 00:05 2024-09-19T07:11:01.758 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-19T07:11:01.758 INFO:teuthology.orchestra.run.smithi159.stdout:Total 71 MB/s | 388 MB 00:05 2024-09-19T07:11:04.884 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-19T07:11:04.967 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-19T07:11:04.968 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-19T07:11:08.240 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-19T07:11:08.241 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-19T07:11:17.498 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-19T07:11:18.300 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-19T07:11:18.390 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-19T07:11:19.191 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: linux-firmware-20240716-145.el9.noarch 2/2 2024-09-19T07:11:19.191 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-19T07:11:19.926 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-19T07:11:19.926 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:11:19.926 INFO:teuthology.orchestra.run.smithi159.stdout:Upgraded: 2024-09-19T07:11:19.927 INFO:teuthology.orchestra.run.smithi159.stdout: linux-firmware-20240905-146.el9.noarch 2024-09-19T07:11:19.927 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:11:19.927 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-19T07:11:20.261 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-19T07:11:20.261 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-19T07:11:20.414 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-19T07:11:20.414 DEBUG:teuthology.orchestra.run.smithi159:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-19T07:11:20.417 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi159.front.sepia.ceph.com to complete... 2024-09-19T07:11:26.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19 07:11:26 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-19T07:11:26.746 INFO:teuthology.orchestra.run.smithi159.stderr:FINISHED --2024-09-19 07:11:26-- 2024-09-19T07:11:26.746 INFO:teuthology.orchestra.run.smithi159.stderr:Total wall clock time: 6.3s 2024-09-19T07:11:26.746 INFO:teuthology.orchestra.run.smithi159.stderr:Downloaded: 1 files, 343M in 6.0s (57.1 MB/s) 2024-09-19T07:11:26.749 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-19T07:11:26.749 DEBUG:teuthology.orchestra.run.smithi159:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-19T07:11:28.813 INFO:teuthology.orchestra.run.smithi159.stdout:Verifying... ######################################## 2024-09-19T07:11:28.870 INFO:teuthology.orchestra.run.smithi159.stdout:Preparing... ######################################## 2024-09-19T07:11:28.929 INFO:teuthology.orchestra.run.smithi159.stdout:Updating / installing... 2024-09-19T07:11:36.979 INFO:teuthology.orchestra.run.smithi159.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-19T07:12:17.205 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi159.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-19T07:12:17.206 DEBUG:teuthology.orchestra.run.smithi159:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-19T07:12:17.290 INFO:teuthology.orchestra.run.smithi159.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-19T07:12:17.290 INFO:teuthology.orchestra.run.smithi159.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-19T07:12:17.290 INFO:teuthology.orchestra.run.smithi159.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-19T07:12:17.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-19T07:12:17.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-19T07:12:17.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-19T07:12:17.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-19T07:12:17.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-19T07:12:17.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-19T07:12:17.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-19T07:12:17.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-19T07:12:17.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-19T07:12:17.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-19T07:12:17.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-19T07:12:17.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-19T07:12:17.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-19T07:12:17.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-19T07:12:17.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-19T07:12:17.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-19T07:12:17.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-19T07:12:17.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-19T07:12:17.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-19T07:12:17.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-19T07:12:17.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-19T07:12:17.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-19T07:12:17.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-19T07:12:17.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-19T07:12:17.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-19T07:12:17.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-19T07:12:17.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-19T07:12:17.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-19T07:12:17.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-19T07:12:17.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-19T07:12:17.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-19T07:12:17.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-19T07:12:17.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-19T07:12:17.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-19T07:12:17.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-19T07:12:17.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-19T07:12:17.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-19T07:12:17.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-19T07:12:17.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-19T07:12:17.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-19T07:12:17.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-19T07:12:17.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-19T07:12:17.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-19T07:12:17.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-19T07:12:17.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-19T07:12:17.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-19T07:12:17.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-19T07:12:17.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-19T07:12:17.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-19T07:12:17.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-19T07:12:17.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-19T07:12:17.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-19T07:12:17.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-19T07:12:17.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-19T07:12:17.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-19T07:12:17.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-19T07:12:17.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-19T07:12:17.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-19T07:12:17.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-19T07:12:17.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-19T07:12:17.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-19T07:12:17.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-19T07:12:17.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-19T07:12:17.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-19T07:12:17.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-19T07:12:17.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-19T07:12:17.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-19T07:12:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-19T07:12:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-19T07:12:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-19T07:12:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-19T07:12:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-19T07:12:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-19T07:12:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-19T07:12:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-19T07:12:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-19T07:12:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-19T07:12:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-19T07:12:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-19T07:12:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-19T07:12:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-19T07:12:17.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-19T07:12:17.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-19T07:12:17.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-19T07:12:17.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-19T07:12:17.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-19T07:12:17.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-19T07:12:17.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-19T07:12:17.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-19T07:12:17.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-19T07:12:17.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-19T07:12:17.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-19T07:12:17.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-19T07:12:17.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-19T07:12:17.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-19T07:12:17.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-19T07:12:17.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-19T07:12:17.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-19T07:12:17.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-19T07:12:17.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-19T07:12:17.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-19T07:12:17.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-19T07:12:17.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-19T07:12:17.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-19T07:12:17.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-19T07:12:17.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-19T07:12:17.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-19T07:12:17.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-19T07:12:17.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-19T07:12:17.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-19T07:12:17.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-19T07:12:17.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-19T07:12:17.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-19T07:12:17.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-19T07:12:17.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-19T07:12:17.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-19T07:12:17.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-19T07:12:17.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-19T07:12:17.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-19T07:12:17.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-19T07:12:17.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-19T07:12:17.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-19T07:12:17.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-19T07:12:17.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-19T07:12:17.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-19T07:12:17.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-19T07:12:17.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-19T07:12:17.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-19T07:12:17.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-19T07:12:17.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-19T07:12:17.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-19T07:12:17.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-19T07:12:17.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-19T07:12:17.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-19T07:12:17.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-19T07:12:17.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-19T07:12:17.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-19T07:12:17.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-19T07:12:17.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-19T07:12:17.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-19T07:12:17.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-19T07:12:17.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-19T07:12:17.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-19T07:12:17.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-19T07:12:17.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-19T07:12:17.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-19T07:12:17.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-19T07:12:17.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-19T07:12:17.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-19T07:12:17.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-19T07:12:17.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-19T07:12:17.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-19T07:12:17.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-19T07:12:17.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-19T07:12:17.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-19T07:12:17.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-19T07:12:17.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-19T07:12:17.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-19T07:12:17.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-19T07:12:17.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-19T07:12:17.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-19T07:12:17.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-19T07:12:17.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-19T07:12:17.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-19T07:12:17.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-19T07:12:17.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-19T07:12:17.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-19T07:12:17.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-19T07:12:17.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-19T07:12:17.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-19T07:12:17.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-19T07:12:17.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-19T07:12:17.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-19T07:12:17.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-19T07:12:17.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-19T07:12:17.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-19T07:12:17.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-19T07:12:17.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-19T07:12:17.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-19T07:12:17.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-19T07:12:17.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-19T07:12:17.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-19T07:12:17.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-19T07:12:17.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-19T07:12:17.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-19T07:12:17.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-19T07:12:17.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-19T07:12:17.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-19T07:12:17.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-19T07:12:17.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-19T07:12:17.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-19T07:12:17.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-19T07:12:17.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-19T07:12:17.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-19T07:12:17.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-19T07:12:17.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-19T07:12:17.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-19T07:12:17.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-19T07:12:17.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-19T07:12:17.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-19T07:12:17.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-19T07:12:17.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-19T07:12:17.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-19T07:12:17.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-19T07:12:17.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-19T07:12:17.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-19T07:12:17.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-19T07:12:17.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-19T07:12:17.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-19T07:12:17.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-19T07:12:17.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-19T07:12:17.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-19T07:12:17.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-19T07:12:17.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-19T07:12:17.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-19T07:12:17.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-19T07:12:17.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-19T07:12:17.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-19T07:12:17.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-19T07:12:17.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-19T07:12:17.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-19T07:12:17.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-19T07:12:17.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-19T07:12:17.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-19T07:12:17.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-19T07:12:17.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-19T07:12:17.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-19T07:12:17.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-19T07:12:17.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-19T07:12:17.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-19T07:12:17.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-19T07:12:17.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-19T07:12:17.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-19T07:12:17.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-19T07:12:17.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-19T07:12:17.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-19T07:12:17.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-19T07:12:17.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-19T07:12:17.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-19T07:12:17.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-19T07:12:17.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-19T07:12:17.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-19T07:12:17.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-19T07:12:17.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-19T07:12:17.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-19T07:12:17.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-19T07:12:17.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-19T07:12:17.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-19T07:12:17.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-19T07:12:17.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-19T07:12:17.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-19T07:12:17.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-19T07:12:17.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-19T07:12:17.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-19T07:12:17.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-19T07:12:17.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-19T07:12:17.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-19T07:12:17.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-19T07:12:17.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-19T07:12:17.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-19T07:12:17.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-19T07:12:17.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-19T07:12:17.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-19T07:12:17.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-19T07:12:17.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-19T07:12:17.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-19T07:12:17.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-19T07:12:17.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-19T07:12:17.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-19T07:12:17.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-19T07:12:17.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-19T07:12:17.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-19T07:12:17.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-19T07:12:17.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-19T07:12:17.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-19T07:12:17.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-19T07:12:17.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-19T07:12:17.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-19T07:12:17.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-19T07:12:17.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-19T07:12:17.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-19T07:12:17.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-19T07:12:17.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-19T07:12:17.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-19T07:12:17.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-19T07:12:17.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-19T07:12:17.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-19T07:12:17.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-19T07:12:17.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-19T07:12:17.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-19T07:12:17.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-19T07:12:17.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-19T07:12:17.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-19T07:12:17.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-19T07:12:17.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-19T07:12:17.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-19T07:12:17.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-19T07:12:17.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-19T07:12:17.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-19T07:12:17.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-19T07:12:17.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-19T07:12:17.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-19T07:12:17.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-19T07:12:17.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-19T07:12:17.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-19T07:12:17.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-19T07:12:17.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-19T07:12:17.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-19T07:12:17.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-19T07:12:17.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-19T07:12:17.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-19T07:12:17.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-19T07:12:17.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-19T07:12:17.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-19T07:12:17.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-19T07:12:17.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-19T07:12:17.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-19T07:12:17.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-19T07:12:17.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-19T07:12:17.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-19T07:12:17.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-19T07:12:17.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-19T07:12:17.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-19T07:12:17.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-19T07:12:17.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-19T07:12:17.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-19T07:12:17.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-19T07:12:17.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-19T07:12:17.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-19T07:12:17.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-19T07:12:17.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-19T07:12:17.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-19T07:12:17.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-19T07:12:17.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-19T07:12:17.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-19T07:12:17.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-19T07:12:17.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-19T07:12:17.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-19T07:12:17.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-19T07:12:17.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-19T07:12:17.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-19T07:12:17.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-19T07:12:17.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-19T07:12:17.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-19T07:12:17.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-19T07:12:17.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-19T07:12:17.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-19T07:12:17.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-19T07:12:17.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-19T07:12:17.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-19T07:12:17.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-19T07:12:17.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-19T07:12:17.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-19T07:12:17.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-19T07:12:17.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-19T07:12:17.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-19T07:12:17.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-19T07:12:17.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-19T07:12:17.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-19T07:12:17.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-19T07:12:17.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-19T07:12:17.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-19T07:12:17.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-19T07:12:17.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-19T07:12:17.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-19T07:12:17.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-19T07:12:17.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-19T07:12:17.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-19T07:12:17.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-19T07:12:17.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-19T07:12:17.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-19T07:12:17.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-19T07:12:17.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-19T07:12:17.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-19T07:12:17.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-19T07:12:17.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-19T07:12:17.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-19T07:12:17.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-19T07:12:17.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-19T07:12:17.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-19T07:12:17.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-19T07:12:17.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-19T07:12:17.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-19T07:12:17.319 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-19T07:12:17.319 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-19T07:12:17.319 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-19T07:12:17.319 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-19T07:12:17.319 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-19T07:12:17.319 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-19T07:12:17.319 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-19T07:12:17.319 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-19T07:12:17.319 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-19T07:12:17.319 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-19T07:12:17.319 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-19T07:12:17.319 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-19T07:12:17.319 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-19T07:12:17.319 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-19T07:12:17.320 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-19T07:12:17.320 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-19T07:12:17.320 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-19T07:12:17.320 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-19T07:12:17.320 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-19T07:12:17.320 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-19T07:12:17.320 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-19T07:12:17.320 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-19T07:12:17.320 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-19T07:12:17.320 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-19T07:12:17.320 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-19T07:12:17.320 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-19T07:12:17.320 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-19T07:12:17.320 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-19T07:12:17.321 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-19T07:12:17.321 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-19T07:12:17.321 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-19T07:12:17.321 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-19T07:12:17.321 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-19T07:12:17.321 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-19T07:12:17.321 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-19T07:12:17.321 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-19T07:12:17.321 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-19T07:12:17.321 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-19T07:12:17.321 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-19T07:12:17.321 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-19T07:12:17.321 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-19T07:12:17.321 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-19T07:12:17.322 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-19T07:12:17.322 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-19T07:12:17.322 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-19T07:12:17.322 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-19T07:12:17.322 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-19T07:12:17.322 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-19T07:12:17.322 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-19T07:12:17.322 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-19T07:12:17.322 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-19T07:12:17.322 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-19T07:12:17.322 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-19T07:12:17.322 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-19T07:12:17.322 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-19T07:12:17.322 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-19T07:12:17.323 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-19T07:12:17.323 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-19T07:12:17.323 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-19T07:12:17.323 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-19T07:12:17.323 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-19T07:12:17.323 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-19T07:12:17.323 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-19T07:12:17.323 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-19T07:12:17.323 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-19T07:12:17.323 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-19T07:12:17.323 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-19T07:12:17.323 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-19T07:12:17.323 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-19T07:12:17.323 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-19T07:12:17.323 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-19T07:12:17.324 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-19T07:12:17.324 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-19T07:12:17.324 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-19T07:12:17.324 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-19T07:12:17.324 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-19T07:12:17.324 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-19T07:12:17.324 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-19T07:12:17.324 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-19T07:12:17.324 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-19T07:12:17.324 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-19T07:12:17.324 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-19T07:12:17.324 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-19T07:12:17.324 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-19T07:12:17.325 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-19T07:12:17.325 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-19T07:12:17.325 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-19T07:12:17.325 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-19T07:12:17.325 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-19T07:12:17.325 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-19T07:12:17.325 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-19T07:12:17.325 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-19T07:12:17.325 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-19T07:12:17.325 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-19T07:12:17.325 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-19T07:12:17.325 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-19T07:12:17.325 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-19T07:12:17.325 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-19T07:12:17.325 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-19T07:12:17.326 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-19T07:12:17.326 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-19T07:12:17.326 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-19T07:12:17.326 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-19T07:12:17.326 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-19T07:12:17.326 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-19T07:12:17.326 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-19T07:12:17.326 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-19T07:12:17.326 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-19T07:12:17.326 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-19T07:12:17.326 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-19T07:12:17.326 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-19T07:12:17.326 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-19T07:12:17.326 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-19T07:12:17.327 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-19T07:12:17.327 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-19T07:12:17.327 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-19T07:12:17.327 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-19T07:12:17.327 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-19T07:12:17.327 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-19T07:12:17.327 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-19T07:12:17.327 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-19T07:12:17.327 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-19T07:12:17.327 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-19T07:12:17.327 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-19T07:12:17.327 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-19T07:12:17.327 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-19T07:12:17.327 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-19T07:12:17.328 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-19T07:12:17.328 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-19T07:12:17.328 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-19T07:12:17.328 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-19T07:12:17.328 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-19T07:12:17.328 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-19T07:12:17.328 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-19T07:12:17.328 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-19T07:12:17.328 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-19T07:12:17.328 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-19T07:12:17.328 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-19T07:12:17.328 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-19T07:12:17.328 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-19T07:12:17.328 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-19T07:12:17.329 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-19T07:12:17.329 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-19T07:12:17.329 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-19T07:12:17.329 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-19T07:12:17.329 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-19T07:12:17.329 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-19T07:12:17.329 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-19T07:12:17.329 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-19T07:12:17.329 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-19T07:12:17.329 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-19T07:12:17.329 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-19T07:12:17.329 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-19T07:12:17.329 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-19T07:12:17.329 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-19T07:12:17.330 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-19T07:12:17.330 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-19T07:12:17.330 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-19T07:12:17.330 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-19T07:12:17.330 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-19T07:12:17.330 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-19T07:12:17.330 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-19T07:12:17.330 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-19T07:12:17.330 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-19T07:12:17.330 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-19T07:12:17.330 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-19T07:12:17.330 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-19T07:12:17.330 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-19T07:12:17.330 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-19T07:12:17.330 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-19T07:12:17.331 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-19T07:12:17.331 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-19T07:12:17.331 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-19T07:12:17.331 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-19T07:12:17.331 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-19T07:12:17.331 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-19T07:12:17.331 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-19T07:12:17.331 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-19T07:12:17.331 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-19T07:12:17.331 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-19T07:12:17.331 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-19T07:12:17.331 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-19T07:12:17.331 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-19T07:12:17.331 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-19T07:12:17.331 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-19T07:12:17.332 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-19T07:12:17.332 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-19T07:12:17.332 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-19T07:12:17.332 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-19T07:12:17.332 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-19T07:12:17.332 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-19T07:12:17.332 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-19T07:12:17.332 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-19T07:12:17.332 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-19T07:12:17.332 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-19T07:12:17.332 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-19T07:12:17.332 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-19T07:12:17.332 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-19T07:12:17.333 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-19T07:12:17.333 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-19T07:12:17.333 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-19T07:12:17.333 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-19T07:12:17.333 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-19T07:12:17.333 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-19T07:12:17.333 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-19T07:12:17.333 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-19T07:12:17.333 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-19T07:12:17.333 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-19T07:12:17.333 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-19T07:12:17.333 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-19T07:12:17.333 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-19T07:12:17.333 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-19T07:12:17.333 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-19T07:12:17.334 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-19T07:12:17.334 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-19T07:12:17.334 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-19T07:12:17.334 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-19T07:12:17.334 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-19T07:12:17.334 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-19T07:12:17.334 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-19T07:12:17.334 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-19T07:12:17.334 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-19T07:12:17.334 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-19T07:12:17.334 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-19T07:12:17.334 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-19T07:12:17.334 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-19T07:12:17.334 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-19T07:12:17.335 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-19T07:12:17.335 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-19T07:12:17.335 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-19T07:12:17.335 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-19T07:12:17.335 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-19T07:12:17.335 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-19T07:12:17.335 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-19T07:12:17.335 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-19T07:12:17.335 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-19T07:12:17.335 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-19T07:12:17.335 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-19T07:12:17.335 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-19T07:12:17.335 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-19T07:12:17.335 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-19T07:12:17.336 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-19T07:12:17.336 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-19T07:12:17.336 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-19T07:12:17.336 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-19T07:12:17.336 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-19T07:12:17.336 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-19T07:12:17.336 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-19T07:12:17.336 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-19T07:12:17.336 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-19T07:12:17.336 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-19T07:12:17.336 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-19T07:12:17.336 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-19T07:12:17.336 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-19T07:12:17.336 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-19T07:12:17.337 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-19T07:12:17.337 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-19T07:12:17.337 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-19T07:12:17.337 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-19T07:12:17.337 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-19T07:12:17.337 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-19T07:12:17.337 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-19T07:12:17.337 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-19T07:12:17.337 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-19T07:12:17.337 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-19T07:12:17.338 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-19T07:12:17.338 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-19T07:12:17.338 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-19T07:12:17.338 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-19T07:12:17.338 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-19T07:12:17.338 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-19T07:12:17.338 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-19T07:12:17.338 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-19T07:12:17.338 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-19T07:12:17.338 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-19T07:12:17.338 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-19T07:12:17.338 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-19T07:12:17.338 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-19T07:12:17.338 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-19T07:12:17.339 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-19T07:12:17.339 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-19T07:12:17.339 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-19T07:12:17.339 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-19T07:12:17.339 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-19T07:12:17.339 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-19T07:12:17.339 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-19T07:12:17.339 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-19T07:12:17.339 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-19T07:12:17.339 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-19T07:12:17.339 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-19T07:12:17.339 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-19T07:12:17.339 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-19T07:12:17.339 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-19T07:12:17.340 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-19T07:12:17.340 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-19T07:12:17.340 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-19T07:12:17.340 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-19T07:12:17.340 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-19T07:12:17.340 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-19T07:12:17.340 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-19T07:12:17.340 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-19T07:12:17.340 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-19T07:12:17.340 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-19T07:12:17.340 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-19T07:12:17.340 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-19T07:12:17.340 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-19T07:12:17.341 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-19T07:12:17.341 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-19T07:12:17.341 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-19T07:12:17.341 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-19T07:12:17.341 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-19T07:12:17.341 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-19T07:12:17.341 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-19T07:12:17.341 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-19T07:12:17.341 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-19T07:12:17.341 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-19T07:12:17.341 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-19T07:12:17.341 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-19T07:12:17.341 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-19T07:12:17.341 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-19T07:12:17.341 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-19T07:12:17.342 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-19T07:12:17.342 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-19T07:12:17.342 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-19T07:12:17.342 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-19T07:12:17.342 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-19T07:12:17.342 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-19T07:12:17.342 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-19T07:12:17.342 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-19T07:12:17.342 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-19T07:12:17.342 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-19T07:12:17.342 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-19T07:12:17.342 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-19T07:12:17.342 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-19T07:12:17.342 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-19T07:12:17.343 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-19T07:12:17.343 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-19T07:12:17.343 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-19T07:12:17.343 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-19T07:12:17.343 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-19T07:12:17.343 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-19T07:12:17.343 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-19T07:12:17.343 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-19T07:12:17.343 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-19T07:12:17.343 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-19T07:12:17.343 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-19T07:12:17.343 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-19T07:12:17.343 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-19T07:12:17.343 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-19T07:12:17.344 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-19T07:12:17.344 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-19T07:12:17.344 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-19T07:12:17.344 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-19T07:12:17.344 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-19T07:12:17.344 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-19T07:12:17.344 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-19T07:12:17.344 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-19T07:12:17.344 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-19T07:12:17.344 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-19T07:12:17.344 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-19T07:12:17.344 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-19T07:12:17.344 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-19T07:12:17.344 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-19T07:12:17.345 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-19T07:12:17.345 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-19T07:12:17.345 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-19T07:12:17.345 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-19T07:12:17.345 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-19T07:12:17.345 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-19T07:12:17.345 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-19T07:12:17.345 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-19T07:12:17.345 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-19T07:12:17.345 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-19T07:12:17.345 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-19T07:12:17.345 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-19T07:12:17.345 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-19T07:12:17.346 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-19T07:12:17.346 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-19T07:12:17.346 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-19T07:12:17.346 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-19T07:12:17.346 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-19T07:12:17.346 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-19T07:12:17.346 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-19T07:12:17.346 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-19T07:12:17.346 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-19T07:12:17.346 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-19T07:12:17.346 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-19T07:12:17.346 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-19T07:12:17.346 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-19T07:12:17.346 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-19T07:12:17.347 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-19T07:12:17.347 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-19T07:12:17.347 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-19T07:12:17.347 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-19T07:12:17.347 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-19T07:12:17.347 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-19T07:12:17.347 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-19T07:12:17.347 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-19T07:12:17.347 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-19T07:12:17.347 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-19T07:12:17.347 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-19T07:12:17.347 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-19T07:12:17.347 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-19T07:12:17.348 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-19T07:12:17.348 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-19T07:12:17.348 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-19T07:12:17.348 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-19T07:12:17.348 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-19T07:12:17.348 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-19T07:12:17.348 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-19T07:12:17.348 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-19T07:12:17.348 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-19T07:12:17.348 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-19T07:12:17.348 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-19T07:12:17.348 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-19T07:12:17.348 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-19T07:12:17.348 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-19T07:12:17.349 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-19T07:12:17.349 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-19T07:12:17.349 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-19T07:12:17.349 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-19T07:12:17.349 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-19T07:12:17.349 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-19T07:12:17.349 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-19T07:12:17.349 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-19T07:12:17.349 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-19T07:12:17.349 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-19T07:12:17.349 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-19T07:12:17.349 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-19T07:12:17.349 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-19T07:12:17.349 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-19T07:12:17.350 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-19T07:12:17.350 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-19T07:12:17.350 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-19T07:12:17.350 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-19T07:12:17.350 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-19T07:12:17.350 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-19T07:12:17.350 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-19T07:12:17.350 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-19T07:12:17.350 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-19T07:12:17.350 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-19T07:12:17.350 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-19T07:12:17.350 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-19T07:12:17.350 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-19T07:12:17.350 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-19T07:12:17.351 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-19T07:12:17.351 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-19T07:12:17.351 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-19T07:12:17.351 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-19T07:12:17.351 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-19T07:12:17.351 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-19T07:12:17.351 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-19T07:12:17.351 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-19T07:12:17.351 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-19T07:12:17.351 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-19T07:12:17.351 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-19T07:12:17.351 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-19T07:12:17.351 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-19T07:12:17.351 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-19T07:12:17.352 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-19T07:12:17.352 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-19T07:12:17.352 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-19T07:12:17.352 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-19T07:12:17.352 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-19T07:12:17.352 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-19T07:12:17.352 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-19T07:12:17.352 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-19T07:12:17.352 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-19T07:12:17.352 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-19T07:12:17.352 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-19T07:12:17.352 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-19T07:12:17.352 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-19T07:12:17.353 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-19T07:12:17.353 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-19T07:12:17.353 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-19T07:12:17.353 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-19T07:12:17.353 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-19T07:12:17.353 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-19T07:12:17.353 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-19T07:12:17.353 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-19T07:12:17.353 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-19T07:12:17.353 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-19T07:12:17.353 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-19T07:12:17.353 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-19T07:12:17.353 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-19T07:12:17.353 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-19T07:12:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-19T07:12:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-19T07:12:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-19T07:12:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-19T07:12:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-19T07:12:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-19T07:12:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-19T07:12:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-19T07:12:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-19T07:12:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-19T07:12:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-19T07:12:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-19T07:12:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-19T07:12:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-19T07:12:17.355 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-19T07:12:17.355 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-19T07:12:17.355 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-19T07:12:17.355 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-19T07:12:17.355 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-19T07:12:17.355 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-19T07:12:17.355 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-19T07:12:17.355 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-19T07:12:17.355 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-19T07:12:17.355 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-19T07:12:17.355 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-19T07:12:17.355 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-19T07:12:17.355 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-19T07:12:17.355 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-19T07:12:17.356 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-19T07:12:17.356 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-19T07:12:17.356 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-19T07:12:17.356 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-19T07:12:17.356 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-19T07:12:17.356 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-19T07:12:17.356 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-19T07:12:17.356 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-19T07:12:17.356 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-19T07:12:17.356 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-19T07:12:17.356 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-19T07:12:17.356 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-19T07:12:17.356 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-19T07:12:17.357 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-19T07:12:17.357 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-19T07:12:17.357 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-19T07:12:17.357 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-19T07:12:17.357 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-19T07:12:17.357 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-19T07:12:17.357 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-19T07:12:17.357 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-19T07:12:17.357 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-19T07:12:17.357 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-19T07:12:17.357 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-19T07:12:17.357 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-19T07:12:17.357 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-19T07:12:17.357 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-19T07:12:17.358 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-19T07:12:17.358 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-19T07:12:17.358 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-19T07:12:17.358 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-19T07:12:17.358 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-19T07:12:17.358 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-19T07:12:17.358 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-19T07:12:17.358 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-19T07:12:17.358 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-19T07:12:17.358 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-19T07:12:17.358 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-19T07:12:17.358 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-19T07:12:17.358 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-19T07:12:17.358 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-19T07:12:17.358 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-19T07:12:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-19T07:12:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-19T07:12:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-19T07:12:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-19T07:12:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-19T07:12:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-19T07:12:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-19T07:12:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-19T07:12:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-19T07:12:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-19T07:12:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-19T07:12:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-19T07:12:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-19T07:12:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-19T07:12:17.359 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-19T07:12:17.360 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-19T07:12:17.360 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-19T07:12:17.360 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-19T07:12:17.360 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-19T07:12:17.360 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-19T07:12:17.360 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-19T07:12:17.360 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-19T07:12:17.360 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-19T07:12:17.360 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-19T07:12:17.360 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-19T07:12:17.360 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-19T07:12:17.360 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-19T07:12:17.360 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-19T07:12:17.360 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-19T07:12:17.361 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-19T07:12:17.361 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-19T07:12:17.361 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-19T07:12:17.361 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-19T07:12:17.361 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-19T07:12:17.361 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-19T07:12:17.361 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-19T07:12:17.361 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-19T07:12:17.361 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-19T07:12:17.361 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-19T07:12:17.361 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-19T07:12:17.361 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-19T07:12:17.361 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-19T07:12:17.361 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-19T07:12:17.362 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-19T07:12:17.362 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-19T07:12:17.362 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-19T07:12:17.362 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-19T07:12:17.362 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-19T07:12:17.362 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-19T07:12:17.362 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-19T07:12:17.362 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-19T07:12:17.362 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-19T07:12:17.362 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-19T07:12:17.362 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-19T07:12:17.362 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-19T07:12:17.363 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-19T07:12:17.363 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-19T07:12:17.363 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-19T07:12:17.363 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-19T07:12:17.363 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-19T07:12:17.363 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-19T07:12:17.363 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-19T07:12:17.363 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-19T07:12:17.363 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-19T07:12:17.363 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-19T07:12:17.363 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-19T07:12:17.363 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-19T07:12:17.363 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-19T07:12:17.363 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-19T07:12:17.364 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-19T07:12:17.364 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-19T07:12:17.364 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-19T07:12:17.364 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-19T07:12:17.364 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-19T07:12:17.364 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-19T07:12:17.364 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-19T07:12:17.364 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-19T07:12:17.364 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-19T07:12:17.364 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-19T07:12:17.364 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-19T07:12:17.364 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-19T07:12:17.364 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-19T07:12:17.364 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-19T07:12:17.364 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-19T07:12:17.365 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-19T07:12:17.365 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-19T07:12:17.365 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-19T07:12:17.365 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-19T07:12:17.365 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-19T07:12:17.365 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-19T07:12:17.365 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-19T07:12:17.365 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-19T07:12:17.365 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-19T07:12:17.365 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-19T07:12:17.365 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-19T07:12:17.365 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-19T07:12:17.365 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-19T07:12:17.365 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-19T07:12:17.366 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-19T07:12:17.366 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-19T07:12:17.366 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-19T07:12:17.366 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-19T07:12:17.366 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-19T07:12:17.366 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-19T07:12:17.366 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-19T07:12:17.366 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-19T07:12:17.366 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-19T07:12:17.366 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-19T07:12:17.366 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-19T07:12:17.366 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-19T07:12:17.366 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-19T07:12:17.366 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-19T07:12:17.366 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-19T07:12:17.367 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-19T07:12:17.367 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-19T07:12:17.367 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-19T07:12:17.367 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-19T07:12:17.367 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-19T07:12:17.367 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-19T07:12:17.367 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-19T07:12:17.367 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-19T07:12:17.367 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-19T07:12:17.367 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-19T07:12:17.367 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-19T07:12:17.367 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-19T07:12:17.367 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-19T07:12:17.367 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-19T07:12:17.367 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-19T07:12:17.368 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-19T07:12:17.368 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-19T07:12:17.368 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-19T07:12:17.368 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-19T07:12:17.368 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-19T07:12:17.368 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-19T07:12:17.368 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-19T07:12:17.368 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-19T07:12:17.368 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-19T07:12:17.368 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-19T07:12:17.368 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-19T07:12:17.368 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-19T07:12:17.368 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-19T07:12:17.368 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-19T07:12:17.369 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-19T07:12:17.369 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-19T07:12:17.369 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-19T07:12:17.369 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-19T07:12:17.369 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-19T07:12:17.369 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-19T07:12:17.369 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-19T07:12:17.369 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-19T07:12:17.369 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-19T07:12:17.369 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-19T07:12:17.369 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-19T07:12:17.369 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-19T07:12:17.369 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-19T07:12:17.369 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-19T07:12:17.370 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-19T07:12:17.370 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-19T07:12:17.370 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-19T07:12:17.370 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-19T07:12:17.370 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-19T07:12:17.370 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-19T07:12:17.370 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-19T07:12:17.370 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-19T07:12:17.370 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-19T07:12:17.370 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-19T07:12:17.370 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-19T07:12:17.370 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-19T07:12:17.370 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-19T07:12:17.370 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-19T07:12:17.370 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-19T07:12:17.371 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-19T07:12:17.371 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-19T07:12:17.371 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-19T07:12:17.371 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-19T07:12:17.371 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-19T07:12:17.371 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-19T07:12:17.371 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-19T07:12:17.371 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-19T07:12:17.371 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-19T07:12:17.371 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-19T07:12:17.371 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-19T07:12:17.371 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-19T07:12:17.371 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-19T07:12:17.371 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-19T07:12:17.372 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-19T07:12:17.372 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-19T07:12:17.372 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-19T07:12:17.372 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-19T07:12:17.372 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-19T07:12:17.372 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-19T07:12:17.372 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-19T07:12:17.372 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-19T07:12:17.372 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-19T07:12:17.372 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-19T07:12:17.372 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-19T07:12:17.372 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-19T07:12:17.372 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-19T07:12:17.372 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-19T07:12:17.373 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-19T07:12:17.373 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-19T07:12:17.373 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-19T07:12:17.373 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-19T07:12:17.373 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-19T07:12:17.373 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-19T07:12:17.373 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-19T07:12:17.373 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-19T07:12:17.373 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-19T07:12:17.373 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-19T07:12:17.373 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-19T07:12:17.373 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-19T07:12:17.373 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-19T07:12:17.373 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-19T07:12:17.374 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-19T07:12:17.374 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-19T07:12:17.374 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-19T07:12:17.374 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-19T07:12:17.374 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-19T07:12:17.374 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-19T07:12:17.374 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-19T07:12:17.374 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-19T07:12:17.374 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-19T07:12:17.374 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-19T07:12:17.374 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-19T07:12:17.374 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-19T07:12:17.374 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-19T07:12:17.374 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-19T07:12:17.374 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-19T07:12:17.375 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-19T07:12:17.375 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-19T07:12:17.375 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-19T07:12:17.375 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-19T07:12:17.375 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-19T07:12:17.375 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-19T07:12:17.375 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-19T07:12:17.375 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-19T07:12:17.375 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-19T07:12:17.375 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-19T07:12:17.375 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-19T07:12:17.375 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-19T07:12:17.375 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-19T07:12:17.375 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-19T07:12:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-19T07:12:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-19T07:12:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-19T07:12:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-19T07:12:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-19T07:12:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-19T07:12:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-19T07:12:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-19T07:12:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-19T07:12:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-19T07:12:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-19T07:12:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-19T07:12:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-19T07:12:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-19T07:12:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-19T07:12:17.377 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-19T07:12:17.377 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-19T07:12:17.377 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-19T07:12:17.377 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-19T07:12:17.377 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-19T07:12:17.377 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-19T07:12:17.377 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-19T07:12:17.377 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-19T07:12:17.377 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-19T07:12:17.377 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-19T07:12:17.377 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-19T07:12:17.377 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-19T07:12:17.377 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-19T07:12:17.377 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-19T07:12:17.378 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-19T07:12:17.378 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-19T07:12:17.378 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-19T07:12:17.378 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-19T07:12:17.378 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-19T07:12:17.378 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-19T07:12:17.378 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-19T07:12:17.378 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-19T07:12:17.378 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-19T07:12:17.378 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-19T07:12:17.378 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-19T07:12:17.378 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-19T07:12:17.378 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-19T07:12:17.378 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-19T07:12:17.378 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-19T07:12:17.379 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-19T07:12:17.379 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-19T07:12:17.379 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-19T07:12:17.379 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-19T07:12:17.379 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-19T07:12:17.379 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-19T07:12:17.379 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-19T07:12:17.379 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-19T07:12:17.379 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-19T07:12:17.379 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-19T07:12:17.379 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-19T07:12:17.379 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-19T07:12:17.379 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-19T07:12:17.379 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-19T07:12:17.380 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-19T07:12:17.380 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-19T07:12:17.380 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-19T07:12:17.380 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-19T07:12:17.380 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-19T07:12:17.380 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-19T07:12:17.380 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-19T07:12:17.380 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-19T07:12:17.380 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-19T07:12:17.380 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-19T07:12:17.380 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-19T07:12:17.380 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-19T07:12:17.380 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-19T07:12:17.380 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-19T07:12:17.381 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-19T07:12:17.381 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-19T07:12:17.381 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-19T07:12:17.381 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-19T07:12:17.381 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-19T07:12:17.381 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-19T07:12:17.381 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-19T07:12:17.381 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-19T07:12:17.381 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-19T07:12:17.381 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-19T07:12:17.381 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-19T07:12:17.381 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-19T07:12:17.381 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-19T07:12:17.381 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-19T07:12:17.382 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-19T07:12:17.382 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-19T07:12:17.382 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-19T07:12:17.382 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-19T07:12:17.382 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-19T07:12:17.382 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-19T07:12:17.382 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-19T07:12:17.382 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-19T07:12:17.382 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-19T07:12:17.382 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-19T07:12:17.382 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-19T07:12:17.382 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-19T07:12:17.382 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-19T07:12:17.382 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-19T07:12:17.383 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-19T07:12:17.383 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-19T07:12:17.383 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-19T07:12:17.383 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-19T07:12:17.383 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-19T07:12:17.383 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-19T07:12:17.383 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-19T07:12:17.383 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-19T07:12:17.383 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-19T07:12:17.383 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-19T07:12:17.383 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-19T07:12:17.383 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-19T07:12:17.383 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-19T07:12:17.383 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-19T07:12:17.384 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-19T07:12:17.384 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-19T07:12:17.384 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-19T07:12:17.384 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-19T07:12:17.384 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-19T07:12:17.384 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-19T07:12:17.384 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-19T07:12:17.384 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-19T07:12:17.384 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-19T07:12:17.385 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-19T07:12:17.385 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-19T07:12:17.385 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-19T07:12:17.385 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-19T07:12:17.385 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-19T07:12:17.385 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-19T07:12:17.385 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-19T07:12:17.385 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-19T07:12:17.385 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-19T07:12:17.385 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-19T07:12:17.385 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-19T07:12:17.386 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-19T07:12:17.386 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-19T07:12:17.386 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-19T07:12:17.386 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-19T07:12:17.386 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-19T07:12:17.386 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-19T07:12:17.386 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-19T07:12:17.386 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-19T07:12:17.386 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-19T07:12:17.386 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-19T07:12:17.386 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-19T07:12:17.387 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-19T07:12:17.387 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-19T07:12:17.387 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-19T07:12:17.387 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-19T07:12:17.387 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-19T07:12:17.387 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-19T07:12:17.387 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-19T07:12:17.387 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-19T07:12:17.387 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-19T07:12:17.387 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-19T07:12:17.387 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-19T07:12:17.387 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-19T07:12:17.388 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-19T07:12:17.388 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-19T07:12:17.388 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-19T07:12:17.388 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-19T07:12:17.388 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-19T07:12:17.388 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-19T07:12:17.388 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-19T07:12:17.388 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-19T07:12:17.388 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-19T07:12:17.388 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-19T07:12:17.388 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-19T07:12:17.389 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-19T07:12:17.389 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-19T07:12:17.389 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-19T07:12:17.389 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-19T07:12:17.389 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-19T07:12:17.389 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-19T07:12:17.389 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-19T07:12:17.389 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-19T07:12:17.389 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-19T07:12:17.389 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-19T07:12:17.389 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-19T07:12:17.390 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-19T07:12:17.390 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-19T07:12:17.390 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-19T07:12:17.390 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-19T07:12:17.390 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-19T07:12:17.390 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-19T07:12:17.390 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-19T07:12:17.390 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-19T07:12:17.390 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-19T07:12:17.390 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-19T07:12:17.390 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-19T07:12:17.390 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-19T07:12:17.391 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-19T07:12:17.391 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-19T07:12:17.391 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-19T07:12:17.391 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-19T07:12:17.391 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-19T07:12:17.391 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-19T07:12:17.391 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-19T07:12:17.391 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-19T07:12:17.391 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-19T07:12:17.391 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-19T07:12:17.391 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-19T07:12:17.392 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-19T07:12:17.392 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-19T07:12:17.392 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-19T07:12:17.392 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-19T07:12:17.392 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-19T07:12:17.392 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-19T07:12:17.392 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-19T07:12:17.392 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-19T07:12:17.392 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-19T07:12:17.392 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-19T07:12:17.392 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-19T07:12:17.392 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-19T07:12:17.393 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-19T07:12:17.393 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-19T07:12:17.393 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-19T07:12:17.393 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-19T07:12:17.393 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-19T07:12:17.393 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-19T07:12:17.393 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-19T07:12:17.393 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-19T07:12:17.393 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-19T07:12:17.393 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-19T07:12:17.394 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-19T07:12:17.394 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-19T07:12:17.394 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-19T07:12:17.394 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-19T07:12:17.394 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-19T07:12:17.394 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-19T07:12:17.394 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-19T07:12:17.394 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-19T07:12:17.394 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-19T07:12:17.394 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-19T07:12:17.395 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-19T07:12:17.395 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-19T07:12:17.395 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-19T07:12:17.395 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-19T07:12:17.395 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-19T07:12:17.395 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-19T07:12:17.395 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-19T07:12:17.395 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-19T07:12:17.395 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-19T07:12:17.395 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-19T07:12:17.395 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-19T07:12:17.396 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-19T07:12:17.396 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-19T07:12:17.396 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-19T07:12:17.396 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-19T07:12:17.396 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-19T07:12:17.396 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-19T07:12:17.396 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-19T07:12:17.396 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-19T07:12:17.396 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-19T07:12:17.396 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-19T07:12:17.396 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-19T07:12:17.397 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-19T07:12:17.397 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-19T07:12:17.397 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-19T07:12:17.397 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-19T07:12:17.397 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-19T07:12:17.397 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-19T07:12:17.397 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-19T07:12:17.397 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-19T07:12:17.397 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-19T07:12:17.397 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-19T07:12:17.398 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-19T07:12:17.398 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-19T07:12:17.398 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-19T07:12:17.398 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-19T07:12:17.398 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-19T07:12:17.398 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-19T07:12:17.398 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-19T07:12:17.398 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-19T07:12:17.398 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-19T07:12:17.398 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-19T07:12:17.398 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-19T07:12:17.399 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-19T07:12:17.399 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-19T07:12:17.399 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-19T07:12:17.399 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-19T07:12:17.399 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-19T07:12:17.399 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-19T07:12:17.399 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-19T07:12:17.399 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-19T07:12:17.399 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-19T07:12:17.399 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-19T07:12:17.399 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-19T07:12:17.399 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-19T07:12:17.400 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-19T07:12:17.400 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-19T07:12:17.400 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-19T07:12:17.400 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-19T07:12:17.400 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-19T07:12:17.400 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-19T07:12:17.400 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-19T07:12:17.400 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-19T07:12:17.400 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-19T07:12:17.400 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-19T07:12:17.400 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-19T07:12:17.401 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-19T07:12:17.401 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-19T07:12:17.401 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-19T07:12:17.401 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-19T07:12:17.401 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-19T07:12:17.401 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-19T07:12:17.401 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-19T07:12:17.401 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-19T07:12:17.401 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-19T07:12:17.401 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-19T07:12:17.401 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-19T07:12:17.401 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-19T07:12:17.402 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-19T07:12:17.402 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-19T07:12:17.402 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-19T07:12:17.402 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-19T07:12:17.402 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-19T07:12:17.402 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-19T07:12:17.402 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-19T07:12:17.402 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-19T07:12:17.402 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-19T07:12:17.402 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-19T07:12:17.402 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-19T07:12:17.402 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-19T07:12:17.403 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-19T07:12:17.403 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-19T07:12:17.403 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-19T07:12:17.403 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-19T07:12:17.403 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-19T07:12:17.403 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-19T07:12:17.403 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-19T07:12:17.403 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-19T07:12:17.403 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-19T07:12:17.403 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-19T07:12:17.403 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-19T07:12:17.403 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-19T07:12:17.403 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-19T07:12:17.404 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-19T07:12:17.404 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-19T07:12:17.404 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-19T07:12:17.404 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-19T07:12:17.404 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-19T07:12:17.404 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-19T07:12:17.404 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-19T07:12:17.404 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-19T07:12:17.404 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-19T07:12:17.404 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-19T07:12:17.404 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-19T07:12:17.404 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-19T07:12:17.405 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-19T07:12:17.405 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-19T07:12:17.405 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-19T07:12:17.405 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-19T07:12:17.405 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-19T07:12:17.405 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-19T07:12:17.405 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-19T07:12:17.405 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-19T07:12:17.405 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-19T07:12:17.405 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-19T07:12:17.405 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-19T07:12:17.405 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-19T07:12:17.406 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-19T07:12:17.406 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-19T07:12:17.406 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-19T07:12:17.406 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-19T07:12:17.406 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-19T07:12:17.406 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-19T07:12:17.406 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-19T07:12:17.406 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-19T07:12:17.406 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-19T07:12:17.406 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-19T07:12:17.406 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-19T07:12:17.407 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-19T07:12:17.407 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-19T07:12:17.407 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-19T07:12:17.407 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-19T07:12:17.407 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-19T07:12:17.407 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-19T07:12:17.407 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-19T07:12:17.407 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-19T07:12:17.407 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-19T07:12:17.407 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-19T07:12:17.407 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-19T07:12:17.407 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-19T07:12:17.408 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-19T07:12:17.408 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-19T07:12:17.408 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-19T07:12:17.408 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-19T07:12:17.408 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-19T07:12:17.408 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-19T07:12:17.408 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-19T07:12:17.408 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-19T07:12:17.408 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-19T07:12:17.408 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-19T07:12:17.408 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-19T07:12:17.408 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-19T07:12:17.409 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-19T07:12:17.409 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-19T07:12:17.409 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-19T07:12:17.409 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-19T07:12:17.409 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-19T07:12:17.409 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-19T07:12:17.409 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-19T07:12:17.409 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-19T07:12:17.409 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-19T07:12:17.409 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-19T07:12:17.409 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-19T07:12:17.409 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-19T07:12:17.410 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-19T07:12:17.410 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-19T07:12:17.410 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-19T07:12:17.410 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-19T07:12:17.410 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-19T07:12:17.410 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-19T07:12:17.410 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-19T07:12:17.410 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-19T07:12:17.410 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-19T07:12:17.410 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-19T07:12:17.410 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-19T07:12:17.411 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-19T07:12:17.411 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-19T07:12:17.411 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-19T07:12:17.411 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-19T07:12:17.411 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-19T07:12:17.411 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-19T07:12:17.411 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-19T07:12:17.411 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-19T07:12:17.411 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-19T07:12:17.411 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-19T07:12:17.411 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-19T07:12:17.411 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-19T07:12:17.412 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-19T07:12:17.412 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-19T07:12:17.412 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-19T07:12:17.412 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-19T07:12:17.412 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-19T07:12:17.412 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-19T07:12:17.412 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-19T07:12:17.412 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-19T07:12:17.412 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-19T07:12:17.412 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-19T07:12:17.412 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-19T07:12:17.412 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-19T07:12:17.413 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-19T07:12:17.413 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-19T07:12:17.413 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-19T07:12:17.413 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-19T07:12:17.413 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-19T07:12:17.413 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-19T07:12:17.413 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-19T07:12:17.413 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-19T07:12:17.413 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-19T07:12:17.413 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-19T07:12:17.413 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-19T07:12:17.413 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-19T07:12:17.414 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-19T07:12:17.414 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-19T07:12:17.414 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-19T07:12:17.414 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-19T07:12:17.414 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-19T07:12:17.414 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-19T07:12:17.414 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-19T07:12:17.414 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-19T07:12:17.414 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-19T07:12:17.414 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-19T07:12:17.414 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-19T07:12:17.414 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-19T07:12:17.415 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-19T07:12:17.415 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-19T07:12:17.415 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-19T07:12:17.415 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-19T07:12:17.415 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-19T07:12:17.415 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-19T07:12:17.415 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-19T07:12:17.415 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-19T07:12:17.415 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-19T07:12:17.415 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-19T07:12:17.415 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-19T07:12:17.415 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-19T07:12:17.416 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-19T07:12:17.416 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-19T07:12:17.416 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-19T07:12:17.416 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-19T07:12:17.416 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-19T07:12:17.416 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-19T07:12:17.416 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-19T07:12:17.416 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-19T07:12:17.416 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-19T07:12:17.416 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-19T07:12:17.416 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-19T07:12:17.417 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-19T07:12:17.417 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-19T07:12:17.417 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-19T07:12:17.417 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-19T07:12:17.417 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-19T07:12:17.417 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-19T07:12:17.417 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-19T07:12:17.417 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-19T07:12:17.417 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-19T07:12:17.417 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-19T07:12:17.417 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-19T07:12:17.417 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-19T07:12:17.418 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-19T07:12:17.418 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-19T07:12:17.418 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-19T07:12:17.418 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-19T07:12:17.418 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-19T07:12:17.418 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-19T07:12:17.418 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-19T07:12:17.418 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-19T07:12:17.418 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-19T07:12:17.418 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-19T07:12:17.418 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-19T07:12:17.418 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-19T07:12:17.419 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-19T07:12:17.419 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-19T07:12:17.419 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-19T07:12:17.419 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-19T07:12:17.419 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-19T07:12:17.419 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-19T07:12:17.419 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-19T07:12:17.419 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-19T07:12:17.419 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-19T07:12:17.420 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-19T07:12:17.420 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-19T07:12:17.420 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-19T07:12:17.420 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-19T07:12:17.420 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-19T07:12:17.420 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-19T07:12:17.420 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-19T07:12:17.420 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-19T07:12:17.420 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-19T07:12:17.420 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-19T07:12:17.420 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-19T07:12:17.421 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-19T07:12:17.421 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-19T07:12:17.421 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-19T07:12:17.421 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-19T07:12:17.421 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-19T07:12:17.421 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-19T07:12:17.421 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-19T07:12:17.421 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-19T07:12:17.421 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-19T07:12:17.421 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-19T07:12:17.421 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-19T07:12:17.421 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-19T07:12:17.422 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-19T07:12:17.422 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-19T07:12:17.422 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-19T07:12:17.422 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-19T07:12:17.422 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-19T07:12:17.422 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-19T07:12:17.422 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-19T07:12:17.422 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-19T07:12:17.422 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-19T07:12:17.422 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-19T07:12:17.422 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-19T07:12:17.422 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-19T07:12:17.422 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-19T07:12:17.423 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-19T07:12:17.423 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-19T07:12:17.423 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-19T07:12:17.423 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-19T07:12:17.423 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-19T07:12:17.423 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-19T07:12:17.423 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-19T07:12:17.423 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-19T07:12:17.423 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-19T07:12:17.423 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-19T07:12:17.423 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-19T07:12:17.424 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-19T07:12:17.424 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-19T07:12:17.424 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-19T07:12:17.424 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-19T07:12:17.424 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-19T07:12:17.424 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-19T07:12:17.424 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-19T07:12:17.424 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-19T07:12:17.424 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-19T07:12:17.424 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-19T07:12:17.424 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-19T07:12:17.425 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-19T07:12:17.425 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-19T07:12:17.425 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-19T07:12:17.425 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-19T07:12:17.425 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-19T07:12:17.425 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-19T07:12:17.425 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-19T07:12:17.425 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-19T07:12:17.425 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-19T07:12:17.425 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-19T07:12:17.425 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-19T07:12:17.425 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-19T07:12:17.426 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-19T07:12:17.426 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-19T07:12:17.426 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-19T07:12:17.426 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-19T07:12:17.426 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-19T07:12:17.426 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-19T07:12:17.426 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-19T07:12:17.426 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-19T07:12:17.426 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-19T07:12:17.426 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-19T07:12:17.426 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-19T07:12:17.426 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-19T07:12:17.426 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-19T07:12:17.426 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-19T07:12:17.427 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-19T07:12:17.427 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-19T07:12:17.427 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-19T07:12:17.427 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-19T07:12:17.427 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-19T07:12:17.427 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-19T07:12:17.427 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-19T07:12:17.427 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-19T07:12:17.427 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-19T07:12:17.427 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-19T07:12:17.427 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-19T07:12:17.427 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-19T07:12:17.427 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-19T07:12:17.428 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-19T07:12:17.428 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-19T07:12:17.428 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-19T07:12:17.428 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-19T07:12:17.428 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-19T07:12:17.428 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-19T07:12:17.428 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-19T07:12:17.428 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-19T07:12:17.428 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-19T07:12:17.428 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-19T07:12:17.428 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-19T07:12:17.429 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-19T07:12:17.429 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-19T07:12:17.429 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-19T07:12:17.429 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-19T07:12:17.429 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-19T07:12:17.429 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-19T07:12:17.429 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-19T07:12:17.429 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-19T07:12:17.429 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-19T07:12:17.429 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-19T07:12:17.429 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-19T07:12:17.430 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-19T07:12:17.430 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-19T07:12:17.430 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-19T07:12:17.430 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-19T07:12:17.430 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-19T07:12:17.430 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-19T07:12:17.430 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-19T07:12:17.430 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-19T07:12:17.430 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-19T07:12:17.430 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-19T07:12:17.430 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-19T07:12:17.430 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-19T07:12:17.431 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-19T07:12:17.431 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-19T07:12:17.431 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-19T07:12:17.431 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-19T07:12:17.431 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-19T07:12:17.431 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-19T07:12:17.431 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-19T07:12:17.431 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-19T07:12:17.431 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-19T07:12:17.431 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-19T07:12:17.431 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-19T07:12:17.431 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-19T07:12:17.432 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-19T07:12:17.432 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-19T07:12:17.432 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-19T07:12:17.432 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-19T07:12:17.432 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-19T07:12:17.432 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-19T07:12:17.432 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-19T07:12:17.432 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-19T07:12:17.432 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-19T07:12:17.432 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-19T07:12:17.432 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-19T07:12:17.433 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-19T07:12:17.433 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-19T07:12:17.433 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-19T07:12:17.433 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-19T07:12:17.433 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-19T07:12:17.433 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-19T07:12:17.433 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-19T07:12:17.433 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-19T07:12:17.433 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-19T07:12:17.433 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-19T07:12:17.433 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-19T07:12:17.433 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-19T07:12:17.434 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-19T07:12:17.434 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-19T07:12:17.434 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-19T07:12:17.434 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-19T07:12:17.434 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-19T07:12:17.434 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-19T07:12:17.434 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-19T07:12:17.434 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-19T07:12:17.434 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-19T07:12:17.434 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-19T07:12:17.434 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-19T07:12:17.435 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-19T07:12:17.435 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-19T07:12:17.435 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-19T07:12:17.435 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-19T07:12:17.435 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-19T07:12:17.435 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-19T07:12:17.435 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-19T07:12:17.435 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-19T07:12:17.435 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-19T07:12:17.435 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-19T07:12:17.435 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-19T07:12:17.435 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-19T07:12:17.436 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-19T07:12:17.436 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-19T07:12:17.436 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-19T07:12:17.436 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-19T07:12:17.436 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-19T07:12:17.436 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-19T07:12:17.436 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-19T07:12:17.436 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-19T07:12:17.436 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-19T07:12:17.436 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-19T07:12:17.437 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-19T07:12:17.437 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-19T07:12:17.437 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-19T07:12:17.437 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-19T07:12:17.437 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-19T07:12:17.437 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-19T07:12:17.437 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-19T07:12:17.437 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-19T07:12:17.437 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-19T07:12:17.437 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-19T07:12:17.437 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-19T07:12:17.438 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-19T07:12:17.438 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-19T07:12:17.438 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-19T07:12:17.438 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-19T07:12:17.438 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-19T07:12:17.438 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-19T07:12:17.438 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-19T07:12:17.438 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-19T07:12:17.438 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-19T07:12:17.438 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-19T07:12:17.438 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-19T07:12:17.438 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-19T07:12:17.439 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-19T07:12:17.439 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-19T07:12:17.439 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-19T07:12:17.439 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-19T07:12:17.439 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-19T07:12:17.439 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-19T07:12:17.439 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-19T07:12:17.439 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-19T07:12:17.439 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-19T07:12:17.439 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-19T07:12:17.439 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-19T07:12:17.439 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-19T07:12:17.440 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-19T07:12:17.440 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-19T07:12:17.440 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-19T07:12:17.440 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-19T07:12:17.440 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-19T07:12:17.440 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-19T07:12:17.440 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-19T07:12:17.440 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-19T07:12:17.440 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-19T07:12:17.440 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-19T07:12:17.441 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-19T07:12:17.441 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-19T07:12:17.441 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-19T07:12:17.441 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-19T07:12:17.446 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T07:12:17.446 DEBUG:teuthology.orchestra.run.smithi159:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-19T07:12:17.472 INFO:teuthology.orchestra.run.smithi159.stdout:postinstall scriptlet (using /bin/sh): 2024-09-19T07:12:17.472 INFO:teuthology.orchestra.run.smithi159.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-19T07:12:17.472 INFO:teuthology.orchestra.run.smithi159.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-19T07:12:17.472 INFO:teuthology.orchestra.run.smithi159.stdout:fi 2024-09-19T07:12:17.472 INFO:teuthology.orchestra.run.smithi159.stdout:for file in vmlinuz System.map config; do 2024-09-19T07:12:17.472 INFO:teuthology.orchestra.run.smithi159.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-19T07:12:17.473 INFO:teuthology.orchestra.run.smithi159.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-19T07:12:17.473 INFO:teuthology.orchestra.run.smithi159.stdout: fi 2024-09-19T07:12:17.473 INFO:teuthology.orchestra.run.smithi159.stdout:done 2024-09-19T07:12:17.473 INFO:teuthology.orchestra.run.smithi159.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-19T07:12:17.473 INFO:teuthology.orchestra.run.smithi159.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-19T07:12:17.473 INFO:teuthology.orchestra.run.smithi159.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-19T07:12:17.473 INFO:teuthology.orchestra.run.smithi159.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-19T07:12:17.473 INFO:teuthology.orchestra.run.smithi159.stdout:fi 2024-09-19T07:12:17.473 INFO:teuthology.orchestra.run.smithi159.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-19T07:12:17.473 INFO:teuthology.orchestra.run.smithi159.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T07:12:17.473 INFO:teuthology.orchestra.run.smithi159.stdout:fi 2024-09-19T07:12:17.473 INFO:teuthology.orchestra.run.smithi159.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-19T07:12:17.474 INFO:teuthology.orchestra.run.smithi159.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-19T07:12:17.474 INFO:teuthology.orchestra.run.smithi159.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T07:12:17.474 INFO:teuthology.orchestra.run.smithi159.stdout:fi 2024-09-19T07:12:17.475 DEBUG:teuthology.orchestra.run.smithi159:> sudo rpm -qi grub2-tools 2024-09-19T07:12:17.551 INFO:teuthology.orchestra.run.smithi159.stdout:Name : grub2-tools 2024-09-19T07:12:17.551 INFO:teuthology.orchestra.run.smithi159.stdout:Epoch : 1 2024-09-19T07:12:17.551 INFO:teuthology.orchestra.run.smithi159.stdout:Version : 2.06 2024-09-19T07:12:17.551 INFO:teuthology.orchestra.run.smithi159.stdout:Release : 82.el9 2024-09-19T07:12:17.551 INFO:teuthology.orchestra.run.smithi159.stdout:Architecture: x86_64 2024-09-19T07:12:17.551 INFO:teuthology.orchestra.run.smithi159.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T07:12:17.551 INFO:teuthology.orchestra.run.smithi159.stdout:Group : Unspecified 2024-09-19T07:12:17.551 INFO:teuthology.orchestra.run.smithi159.stdout:Size : 8274670 2024-09-19T07:12:17.551 INFO:teuthology.orchestra.run.smithi159.stdout:License : GPLv3+ 2024-09-19T07:12:17.552 INFO:teuthology.orchestra.run.smithi159.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T07:12:17.552 INFO:teuthology.orchestra.run.smithi159.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T07:12:17.552 INFO:teuthology.orchestra.run.smithi159.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T07:12:17.552 INFO:teuthology.orchestra.run.smithi159.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T07:12:17.552 INFO:teuthology.orchestra.run.smithi159.stdout:Packager : builder@centos.org 2024-09-19T07:12:17.552 INFO:teuthology.orchestra.run.smithi159.stdout:Vendor : CentOS 2024-09-19T07:12:17.552 INFO:teuthology.orchestra.run.smithi159.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T07:12:17.552 INFO:teuthology.orchestra.run.smithi159.stdout:Summary : Support tools for GRUB. 2024-09-19T07:12:17.552 INFO:teuthology.orchestra.run.smithi159.stdout:Description : 2024-09-19T07:12:17.552 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:12:17.552 INFO:teuthology.orchestra.run.smithi159.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T07:12:17.552 INFO:teuthology.orchestra.run.smithi159.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T07:12:17.552 INFO:teuthology.orchestra.run.smithi159.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T07:12:17.552 INFO:teuthology.orchestra.run.smithi159.stdout:hardware devices. 2024-09-19T07:12:17.553 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:12:17.553 INFO:teuthology.orchestra.run.smithi159.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T07:12:17.553 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T07:12:17.553 INFO:teuthology.task.kernel:Updating grub on smithi159 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T07:12:17.554 DEBUG:teuthology.orchestra.run.smithi159:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T07:12:18.211 INFO:teuthology.orchestra.run.smithi159.stderr:Generating grub configuration file ... 2024-09-19T07:12:19.640 INFO:teuthology.orchestra.run.smithi159.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T07:12:19.676 INFO:teuthology.orchestra.run.smithi159.stderr:done 2024-09-19T07:12:19.681 DEBUG:teuthology.orchestra.run.smithi159:> mktemp 2024-09-19T07:12:19.736 INFO:teuthology.orchestra.run.smithi159.stdout:/tmp/tmp.wn4JMZxWl7 2024-09-19T07:12:19.737 DEBUG:teuthology.orchestra.run.smithi159:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wn4JMZxWl7 2024-09-19T07:12:19.802 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 0666 /tmp/tmp.wn4JMZxWl7 2024-09-19T07:12:19.942 DEBUG:teuthology.orchestra.remote:smithi159:/tmp/tmp.wn4JMZxWl7 is 6KB 2024-09-19T07:12:19.990 DEBUG:teuthology.orchestra.run.smithi159:> rm -fr /tmp/tmp.wn4JMZxWl7 2024-09-19T07:12:20.006 DEBUG:teuthology.orchestra.run.smithi159:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T07:12:20.068 INFO:teuthology.orchestra.run.smithi159.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T07:12:20.068 INFO:teuthology.orchestra.run.smithi159.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T07:12:20.068 INFO:teuthology.orchestra.run.smithi159.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T07:12:20.068 INFO:teuthology.orchestra.run.smithi159.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-19T07:12:20.069 DEBUG:teuthology.orchestra.run.smithi159:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-19T07:12:20.204 DEBUG:teuthology.orchestra.run.smithi159:> sudo shutdown -r now 2024-09-19T07:12:50.206 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-19T07:12:50.207 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-19T07:12:50.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:13:24.804 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-09-19T07:13:33.806 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-19T07:13:33.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:14:33.810 DEBUG:teuthology.orchestra.remote:timed out 2024-09-19T07:14:45.811 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-19T07:14:45.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:14:46.159 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-09-19T07:14:46.655 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-19T07:14:46.655 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-19T07:14:46.655 DEBUG:teuthology.orchestra.run.smithi159:> uname -r 2024-09-19T07:14:46.690 INFO:teuthology.orchestra.run.smithi159.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-19T07:14:46.690 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-19T07:14:46.690 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T07:14:46.690 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T07:14:47.691 DEBUG:teuthology.parallel:result is None 2024-09-19T07:14:47.691 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T07:14:47.700 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T07:14:47.700 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T07:14:47.704 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T07:14:47.707 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T07:14:47.737 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T07:14:47.743 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T07:14:47.770 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T07:14:47.771 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T07:14:47.781 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T07:14:47.783 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T07:14:47.916 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T07:14:47.923 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T07:14:47.923 DEBUG:teuthology.orchestra.run.smithi189:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T07:14:47.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:14:47.940 DEBUG:teuthology.orchestra.run.smithi203:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T07:14:47.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:14:47.958 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T07:14:47.962 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T07:14:47.983 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T07:14:48.018 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:14:48.037 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:14:48.044 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:14:48.057 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:14:48.058 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:14:48.086 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:14:48.089 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T07:14:48.096 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T07:14:48.096 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T07:14:48.100 DEBUG:teuthology.orchestra.run.smithi189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T07:14:48.102 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T07:14:48.152 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T07:14:48.161 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T07:14:48.161 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T07:14:48.165 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T07:14:48.169 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T07:14:48.196 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:14:48.296 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:14:48.389 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:14:48.452 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:14:48.522 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:14:48.522 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T07:14:48.576 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:14:48.622 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:14:48.671 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:14:48.757 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:14:48.801 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-19T07:14:48.801 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T07:14:48.868 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:14:48.914 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:14:48.964 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:14:49.048 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:14:49.095 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-19T07:14:49.095 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T07:14:49.163 DEBUG:teuthology.orchestra.run.smithi159:> sudo service rsyslog restart 2024-09-19T07:14:49.167 DEBUG:teuthology.orchestra.run.smithi189:> sudo service rsyslog restart 2024-09-19T07:14:49.170 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2024-09-19T07:14:49.215 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:14:49.242 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:14:49.245 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:14:49.687 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T07:14:49.694 INFO:teuthology.task.internal:Starting timer... 2024-09-19T07:14:49.694 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T07:14:49.748 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T07:14:49.780 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T07:14:49.780 DEBUG:teuthology.orchestra.run.smithi159:> sudo service auditd rotate 2024-09-19T07:14:49.863 INFO:teuthology.orchestra.run.smithi159.stdout:Rotating logs: 2024-09-19T07:14:49.866 DEBUG:teuthology.orchestra.run.smithi189:> sudo service auditd rotate 2024-09-19T07:14:49.922 INFO:teuthology.orchestra.run.smithi189.stdout:Rotating logs: 2024-09-19T07:14:49.924 DEBUG:teuthology.orchestra.run.smithi203:> sudo service auditd rotate 2024-09-19T07:14:49.982 INFO:teuthology.orchestra.run.smithi203.stdout:Rotating logs: 2024-09-19T07:14:49.984 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T07:14:49.984 DEBUG:teuthology.orchestra.run.smithi159:> /usr/sbin/getenforce 2024-09-19T07:14:50.023 INFO:teuthology.orchestra.run.smithi159.stdout:Permissive 2024-09-19T07:14:50.023 DEBUG:teuthology.orchestra.run.smithi189:> /usr/sbin/getenforce 2024-09-19T07:14:50.048 INFO:teuthology.orchestra.run.smithi189.stdout:Permissive 2024-09-19T07:14:50.048 DEBUG:teuthology.orchestra.run.smithi203:> /usr/sbin/getenforce 2024-09-19T07:14:50.073 INFO:teuthology.orchestra.run.smithi203.stdout:Permissive 2024-09-19T07:14:50.073 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi159.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi189.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi203.front.sepia.ceph.com': 'permissive'} 2024-09-19T07:14:50.073 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:14:50.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:14:50.131 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:14:50.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:14:50.160 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:14:50.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:14:50.191 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T07:14:50.191 DEBUG:teuthology.orchestra.run.smithi159:> sudo /usr/sbin/setenforce permissive 2024-09-19T07:14:50.257 DEBUG:teuthology.orchestra.run.smithi189:> sudo /usr/sbin/setenforce permissive 2024-09-19T07:14:50.300 DEBUG:teuthology.orchestra.run.smithi203:> sudo /usr/sbin/setenforce permissive 2024-09-19T07:14:50.338 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T07:14:50.349 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-19T07:14:50.353 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-19T07:14:50.464 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T07:14:50.475 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T07:14:50.476 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi159.front.sepia.ceph.com,smithi189.front.sepia.ceph.com,smithi203.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T07:19:24.937 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi159.front.sepia.ceph.com'), Remote(name='ubuntu@smithi189.front.sepia.ceph.com'), Remote(name='ubuntu@smithi203.front.sepia.ceph.com')] 2024-09-19T07:19:24.938 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-19T07:19:24.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:19:25.029 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-09-19T07:19:25.134 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-19T07:19:25.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-19T07:19:25.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:19:25.231 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-09-19T07:19:25.316 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-19T07:19:25.317 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-19T07:19:25.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:19:25.396 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-09-19T07:19:25.481 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-09-19T07:19:25.482 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T07:19:25.493 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T07:19:25.493 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T07:19:25.494 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:19:25.497 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T07:19:25.498 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:19:25.500 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T07:19:25.500 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:19:25.528 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T07:19:25.543 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T07:19:25.551 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T07:19:25.555 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T07:19:25.566 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T07:19:25.568 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: ntpd: command not found 2024-09-19T07:19:25.579 INFO:teuthology.orchestra.run.smithi189.stdout:506 Cannot talk to daemon 2024-09-19T07:19:25.585 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T07:19:25.591 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: ntpd: command not found 2024-09-19T07:19:25.594 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T07:19:25.602 INFO:teuthology.orchestra.run.smithi203.stdout:506 Cannot talk to daemon 2024-09-19T07:19:25.608 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T07:19:25.616 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T07:19:25.630 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T07:19:25.636 INFO:teuthology.orchestra.run.smithi159.stderr:sudo: ntpd: command not found 2024-09-19T07:19:25.660 INFO:teuthology.orchestra.run.smithi159.stdout:506 Cannot talk to daemon 2024-09-19T07:19:25.676 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:19:25.680 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:19:25.680 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-09-19T07:19:25.680 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:19:25.680 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:19:25.680 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:19:25.680 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:19:25.687 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:19:25.689 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T07:19:25.691 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:19:25.691 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-09-19T07:19:25.691 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:19:25.691 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:19:25.691 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:19:25.691 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:19:25.719 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T07:19:25.812 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:19:25.819 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:19:25.819 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-09-19T07:19:25.819 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:19:25.819 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:19:25.819 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:19:25.819 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:19:25.820 INFO:teuthology.run_tasks:Running task install... 2024-09-19T07:19:25.829 DEBUG:teuthology.task.install:project ceph 2024-09-19T07:19:25.829 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T07:19:25.829 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T07:19:25.829 INFO:teuthology.task.install:Using flavor: default 2024-09-19T07:19:25.834 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T07:19:25.834 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T07:19:25.835 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T07:19:25.835 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:19:25.837 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T07:19:25.837 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:19:25.838 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T07:19:25.838 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:19:25.984 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T07:19:25.984 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T07:19:25.987 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T07:19:25.987 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T07:19:25.994 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T07:19:25.995 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T07:19:26.093 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T07:19:26.093 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-19T07:19:26.093 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T07:19:26.107 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T07:19:26.107 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-19T07:19:26.107 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T07:19:26.119 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T07:19:26.119 DEBUG:teuthology.orchestra.run.smithi203:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T07:19:26.134 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T07:19:26.134 DEBUG:teuthology.orchestra.run.smithi189:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T07:19:26.179 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T07:19:26.179 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:19:26.179 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T07:19:26.184 DEBUG:teuthology.orchestra.run.smithi203:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T07:19:26.200 DEBUG:teuthology.orchestra.run.smithi189:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T07:19:26.235 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T07:19:26.235 DEBUG:teuthology.orchestra.run.smithi159:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T07:19:26.259 DEBUG:teuthology.orchestra.run.smithi203:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T07:19:26.275 DEBUG:teuthology.orchestra.run.smithi189:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T07:19:26.324 INFO:teuthology.orchestra.run.smithi203.stdout:check_obsoletes = 1 2024-09-19T07:19:26.325 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum clean all 2024-09-19T07:19:26.333 DEBUG:teuthology.orchestra.run.smithi159:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T07:19:26.340 INFO:teuthology.orchestra.run.smithi189.stdout:check_obsoletes = 1 2024-09-19T07:19:26.341 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum clean all 2024-09-19T07:19:26.411 DEBUG:teuthology.orchestra.run.smithi159:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T07:19:26.466 INFO:teuthology.orchestra.run.smithi159.stdout:check_obsoletes = 1 2024-09-19T07:19:26.469 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum clean all 2024-09-19T07:19:26.550 INFO:teuthology.orchestra.run.smithi203.stdout:45 files removed 2024-09-19T07:19:26.566 INFO:teuthology.orchestra.run.smithi189.stdout:45 files removed 2024-09-19T07:19:26.577 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T07:19:26.593 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T07:19:26.754 INFO:teuthology.orchestra.run.smithi159.stdout:45 files removed 2024-09-19T07:19:26.787 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T07:19:27.132 INFO:teuthology.orchestra.run.smithi203.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-19T07:19:27.133 INFO:teuthology.orchestra.run.smithi189.stdout:ceph packages for x86_64 316 kB/s | 84 kB 00:00 2024-09-19T07:19:27.408 INFO:teuthology.orchestra.run.smithi189.stdout:ceph noarch packages 51 kB/s | 12 kB 00:00 2024-09-19T07:19:27.425 INFO:teuthology.orchestra.run.smithi203.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-19T07:19:27.460 INFO:teuthology.orchestra.run.smithi159.stdout:ceph packages for x86_64 309 kB/s | 84 kB 00:00 2024-09-19T07:19:27.684 INFO:teuthology.orchestra.run.smithi189.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-19T07:19:27.700 INFO:teuthology.orchestra.run.smithi203.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-19T07:19:27.760 INFO:teuthology.orchestra.run.smithi159.stdout:ceph noarch packages 46 kB/s | 12 kB 00:00 2024-09-19T07:19:28.036 INFO:teuthology.orchestra.run.smithi159.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-19T07:19:28.636 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-19T07:19:30.342 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.3 MB 00:02 2024-09-19T07:19:30.775 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.3 MB 00:03 2024-09-19T07:19:31.803 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 20 MB 00:02 2024-09-19T07:19:32.885 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 19 MB/s | 20 MB 00:01 2024-09-19T07:19:33.868 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 8.3 MB/s | 20 MB 00:02 2024-09-19T07:19:36.172 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-19T07:19:37.328 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-19T07:19:37.714 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 19 kB 00:00 2024-09-19T07:19:38.069 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-19T07:19:38.852 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-19T07:19:38.945 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 19 kB 00:00 2024-09-19T07:19:39.544 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 19 kB 00:00 2024-09-19T07:19:39.858 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-19T07:19:40.474 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-19T07:19:45.248 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-19T07:19:45.249 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-19T07:19:45.862 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-09-19T07:19:47.516 INFO:teuthology.orchestra.run.smithi203.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T07:19:47.517 INFO:teuthology.orchestra.run.smithi203.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T07:19:47.528 INFO:teuthology.orchestra.run.smithi203.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T07:19:47.529 INFO:teuthology.orchestra.run.smithi203.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T07:19:47.530 INFO:teuthology.orchestra.run.smithi203.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T07:19:47.531 INFO:teuthology.orchestra.run.smithi203.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T07:19:47.532 INFO:teuthology.orchestra.run.smithi203.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T07:19:47.535 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T07:19:47.536 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T07:19:47.598 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-09-19T07:19:47.611 INFO:teuthology.orchestra.run.smithi203.stdout:====================================================================================== 2024-09-19T07:19:47.612 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-09-19T07:19:47.612 INFO:teuthology.orchestra.run.smithi203.stdout:====================================================================================== 2024-09-19T07:19:47.612 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-09-19T07:19:47.612 INFO:teuthology.orchestra.run.smithi203.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T07:19:47.612 INFO:teuthology.orchestra.run.smithi203.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T07:19:47.612 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T07:19:47.612 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T07:19:47.612 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T07:19:47.612 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T07:19:47.612 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T07:19:47.612 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T07:19:47.612 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T07:19:47.613 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T07:19:47.613 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T07:19:47.613 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T07:19:47.613 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T07:19:47.613 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T07:19:47.613 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T07:19:47.613 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T07:19:47.613 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T07:19:47.613 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T07:19:47.613 INFO:teuthology.orchestra.run.smithi203.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T07:19:47.613 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T07:19:47.613 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T07:19:47.613 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T07:19:47.614 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T07:19:47.614 INFO:teuthology.orchestra.run.smithi203.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T07:19:47.614 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T07:19:47.614 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T07:19:47.614 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T07:19:47.614 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T07:19:47.614 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T07:19:47.614 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T07:19:47.614 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T07:19:47.614 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T07:19:47.614 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T07:19:47.614 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T07:19:47.615 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T07:19:47.615 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T07:19:47.615 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T07:19:47.615 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrading: 2024-09-19T07:19:47.615 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T07:19:47.615 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T07:19:47.615 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T07:19:47.615 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T07:19:47.615 INFO:teuthology.orchestra.run.smithi203.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T07:19:47.615 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T07:19:47.615 INFO:teuthology.orchestra.run.smithi203.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T07:19:47.615 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T07:19:47.615 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-09-19T07:19:47.616 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T07:19:47.616 INFO:teuthology.orchestra.run.smithi203.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T07:19:47.616 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T07:19:47.616 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T07:19:47.616 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T07:19:47.616 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T07:19:47.616 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T07:19:47.616 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T07:19:47.616 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T07:19:47.616 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T07:19:47.616 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T07:19:47.616 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T07:19:47.617 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T07:19:47.617 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T07:19:47.617 INFO:teuthology.orchestra.run.smithi203.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T07:19:47.617 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T07:19:47.617 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T07:19:47.617 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T07:19:47.617 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T07:19:47.617 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T07:19:47.617 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T07:19:47.617 INFO:teuthology.orchestra.run.smithi203.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T07:19:47.617 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T07:19:47.617 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T07:19:47.617 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T07:19:47.617 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T07:19:47.618 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T07:19:47.618 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T07:19:47.618 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T07:19:47.618 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T07:19:47.618 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T07:19:47.618 INFO:teuthology.orchestra.run.smithi203.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T07:19:47.618 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T07:19:47.618 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T07:19:47.618 INFO:teuthology.orchestra.run.smithi203.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T07:19:47.618 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T07:19:47.618 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T07:19:47.618 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T07:19:47.618 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T07:19:47.618 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T07:19:47.619 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T07:19:47.619 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T07:19:47.619 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T07:19:47.619 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T07:19:47.619 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T07:19:47.619 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T07:19:47.619 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T07:19:47.619 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T07:19:47.619 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T07:19:47.619 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T07:19:47.619 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T07:19:47.619 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T07:19:47.619 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T07:19:47.619 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T07:19:47.620 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T07:19:47.620 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T07:19:47.620 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T07:19:47.620 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T07:19:47.620 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T07:19:47.620 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T07:19:47.620 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T07:19:47.620 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T07:19:47.620 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T07:19:47.620 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T07:19:47.620 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T07:19:47.620 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T07:19:47.620 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T07:19:47.621 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T07:19:47.621 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T07:19:47.621 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T07:19:47.621 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T07:19:47.621 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T07:19:47.621 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T07:19:47.621 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T07:19:47.621 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T07:19:47.621 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T07:19:47.621 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T07:19:47.621 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T07:19:47.621 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T07:19:47.621 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T07:19:47.622 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T07:19:47.622 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T07:19:47.622 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T07:19:47.622 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T07:19:47.622 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T07:19:47.622 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T07:19:47.622 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T07:19:47.622 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T07:19:47.622 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T07:19:47.623 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T07:19:47.623 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T07:19:47.623 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T07:19:47.623 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T07:19:47.623 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T07:19:47.623 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T07:19:47.623 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T07:19:47.623 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T07:19:47.623 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T07:19:47.623 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T07:19:47.623 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T07:19:47.623 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T07:19:47.623 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T07:19:47.623 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T07:19:47.624 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T07:19:47.624 INFO:teuthology.orchestra.run.smithi203.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T07:19:47.624 INFO:teuthology.orchestra.run.smithi203.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T07:19:47.624 INFO:teuthology.orchestra.run.smithi203.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T07:19:47.624 INFO:teuthology.orchestra.run.smithi203.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T07:19:47.624 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T07:19:47.624 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T07:19:47.624 INFO:teuthology.orchestra.run.smithi203.stdout:Installing weak dependencies: 2024-09-19T07:19:47.624 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T07:19:47.624 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:19:47.624 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-09-19T07:19:47.624 INFO:teuthology.orchestra.run.smithi203.stdout:====================================================================================== 2024-09-19T07:19:47.624 INFO:teuthology.orchestra.run.smithi203.stdout:Install 148 Packages 2024-09-19T07:19:47.624 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrade 8 Packages 2024-09-19T07:19:47.625 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:19:47.625 INFO:teuthology.orchestra.run.smithi159.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T07:19:47.625 INFO:teuthology.orchestra.run.smithi159.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T07:19:47.626 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 226 M 2024-09-19T07:19:47.626 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-09-19T07:19:47.636 INFO:teuthology.orchestra.run.smithi159.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T07:19:47.637 INFO:teuthology.orchestra.run.smithi159.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T07:19:47.638 INFO:teuthology.orchestra.run.smithi159.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T07:19:47.639 INFO:teuthology.orchestra.run.smithi159.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T07:19:47.641 INFO:teuthology.orchestra.run.smithi159.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T07:19:47.644 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T07:19:47.645 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T07:19:47.713 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-19T07:19:47.735 INFO:teuthology.orchestra.run.smithi159.stdout:====================================================================================== 2024-09-19T07:19:47.735 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-09-19T07:19:47.736 INFO:teuthology.orchestra.run.smithi159.stdout:====================================================================================== 2024-09-19T07:19:47.736 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-19T07:19:47.736 INFO:teuthology.orchestra.run.smithi159.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T07:19:47.736 INFO:teuthology.orchestra.run.smithi159.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T07:19:47.736 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T07:19:47.736 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T07:19:47.736 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T07:19:47.736 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T07:19:47.736 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T07:19:47.736 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T07:19:47.736 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T07:19:47.736 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T07:19:47.737 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T07:19:47.737 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T07:19:47.737 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T07:19:47.737 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T07:19:47.737 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T07:19:47.737 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T07:19:47.737 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T07:19:47.737 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T07:19:47.737 INFO:teuthology.orchestra.run.smithi159.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T07:19:47.737 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T07:19:47.737 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T07:19:47.737 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T07:19:47.737 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T07:19:47.738 INFO:teuthology.orchestra.run.smithi159.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T07:19:47.738 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T07:19:47.738 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T07:19:47.738 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T07:19:47.738 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T07:19:47.738 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T07:19:47.738 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T07:19:47.738 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T07:19:47.738 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T07:19:47.738 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T07:19:47.738 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T07:19:47.738 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T07:19:47.738 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T07:19:47.738 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T07:19:47.739 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrading: 2024-09-19T07:19:47.739 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T07:19:47.739 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T07:19:47.739 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T07:19:47.739 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T07:19:47.739 INFO:teuthology.orchestra.run.smithi159.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T07:19:47.739 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T07:19:47.739 INFO:teuthology.orchestra.run.smithi159.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T07:19:47.739 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T07:19:47.739 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-09-19T07:19:47.740 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T07:19:47.740 INFO:teuthology.orchestra.run.smithi159.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T07:19:47.740 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T07:19:47.740 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T07:19:47.740 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T07:19:47.740 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T07:19:47.740 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T07:19:47.740 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T07:19:47.740 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T07:19:47.740 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T07:19:47.740 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T07:19:47.740 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T07:19:47.740 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T07:19:47.741 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T07:19:47.741 INFO:teuthology.orchestra.run.smithi159.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T07:19:47.741 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T07:19:47.741 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T07:19:47.741 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T07:19:47.741 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T07:19:47.741 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T07:19:47.742 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T07:19:47.742 INFO:teuthology.orchestra.run.smithi159.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T07:19:47.742 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T07:19:47.742 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T07:19:47.742 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T07:19:47.742 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T07:19:47.742 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T07:19:47.742 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T07:19:47.742 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T07:19:47.742 INFO:teuthology.orchestra.run.smithi159.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T07:19:47.742 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T07:19:47.742 INFO:teuthology.orchestra.run.smithi159.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T07:19:47.743 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T07:19:47.743 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T07:19:47.743 INFO:teuthology.orchestra.run.smithi159.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T07:19:47.743 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T07:19:47.743 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T07:19:47.743 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T07:19:47.743 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T07:19:47.743 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T07:19:47.743 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T07:19:47.743 INFO:teuthology.orchestra.run.smithi159.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T07:19:47.743 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T07:19:47.743 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T07:19:47.744 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T07:19:47.744 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T07:19:47.744 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T07:19:47.744 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T07:19:47.744 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T07:19:47.744 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T07:19:47.744 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T07:19:47.744 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T07:19:47.744 INFO:teuthology.orchestra.run.smithi159.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T07:19:47.744 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T07:19:47.744 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T07:19:47.744 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T07:19:47.744 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T07:19:47.745 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T07:19:47.745 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T07:19:47.745 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T07:19:47.745 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T07:19:47.745 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T07:19:47.745 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T07:19:47.745 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T07:19:47.745 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T07:19:47.745 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T07:19:47.745 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T07:19:47.745 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T07:19:47.745 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T07:19:47.745 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T07:19:47.745 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T07:19:47.746 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T07:19:47.746 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T07:19:47.746 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T07:19:47.746 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T07:19:47.746 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T07:19:47.746 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T07:19:47.746 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T07:19:47.746 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T07:19:47.746 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T07:19:47.746 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T07:19:47.746 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T07:19:47.747 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T07:19:47.747 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T07:19:47.747 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T07:19:47.747 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T07:19:47.747 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T07:19:47.747 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T07:19:47.747 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T07:19:47.747 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T07:19:47.747 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T07:19:47.747 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T07:19:47.747 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T07:19:47.747 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T07:19:47.748 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T07:19:47.748 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T07:19:47.748 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T07:19:47.748 INFO:teuthology.orchestra.run.smithi159.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T07:19:47.748 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T07:19:47.748 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T07:19:47.748 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T07:19:47.748 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T07:19:47.748 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T07:19:47.748 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T07:19:47.749 INFO:teuthology.orchestra.run.smithi159.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T07:19:47.749 INFO:teuthology.orchestra.run.smithi159.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T07:19:47.749 INFO:teuthology.orchestra.run.smithi159.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T07:19:47.749 INFO:teuthology.orchestra.run.smithi159.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T07:19:47.749 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T07:19:47.749 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T07:19:47.749 INFO:teuthology.orchestra.run.smithi159.stdout:Installing weak dependencies: 2024-09-19T07:19:47.749 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T07:19:47.749 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:19:47.749 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-19T07:19:47.749 INFO:teuthology.orchestra.run.smithi159.stdout:====================================================================================== 2024-09-19T07:19:47.749 INFO:teuthology.orchestra.run.smithi159.stdout:Install 148 Packages 2024-09-19T07:19:47.750 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrade 8 Packages 2024-09-19T07:19:47.750 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:19:47.750 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 226 M 2024-09-19T07:19:47.750 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-19T07:19:47.991 INFO:teuthology.orchestra.run.smithi189.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T07:19:47.992 INFO:teuthology.orchestra.run.smithi189.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T07:19:48.005 INFO:teuthology.orchestra.run.smithi189.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T07:19:48.006 INFO:teuthology.orchestra.run.smithi189.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T07:19:48.007 INFO:teuthology.orchestra.run.smithi189.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T07:19:48.008 INFO:teuthology.orchestra.run.smithi189.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T07:19:48.010 INFO:teuthology.orchestra.run.smithi189.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T07:19:48.012 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T07:19:48.013 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T07:19:48.076 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-19T07:19:48.095 INFO:teuthology.orchestra.run.smithi189.stdout:====================================================================================== 2024-09-19T07:19:48.096 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-19T07:19:48.096 INFO:teuthology.orchestra.run.smithi189.stdout:====================================================================================== 2024-09-19T07:19:48.096 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-19T07:19:48.096 INFO:teuthology.orchestra.run.smithi189.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T07:19:48.096 INFO:teuthology.orchestra.run.smithi189.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T07:19:48.096 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T07:19:48.096 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T07:19:48.096 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T07:19:48.096 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T07:19:48.096 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T07:19:48.096 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T07:19:48.096 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T07:19:48.096 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T07:19:48.097 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T07:19:48.097 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T07:19:48.097 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T07:19:48.097 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T07:19:48.097 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T07:19:48.097 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T07:19:48.097 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T07:19:48.097 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T07:19:48.097 INFO:teuthology.orchestra.run.smithi189.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T07:19:48.097 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T07:19:48.097 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T07:19:48.097 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T07:19:48.097 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T07:19:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T07:19:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T07:19:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T07:19:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T07:19:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T07:19:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T07:19:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T07:19:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T07:19:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T07:19:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T07:19:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T07:19:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T07:19:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T07:19:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T07:19:48.099 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrading: 2024-09-19T07:19:48.099 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T07:19:48.099 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T07:19:48.099 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T07:19:48.099 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T07:19:48.099 INFO:teuthology.orchestra.run.smithi189.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T07:19:48.099 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T07:19:48.099 INFO:teuthology.orchestra.run.smithi189.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T07:19:48.099 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T07:19:48.099 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-19T07:19:48.099 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T07:19:48.099 INFO:teuthology.orchestra.run.smithi189.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T07:19:48.099 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T07:19:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T07:19:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T07:19:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T07:19:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T07:19:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T07:19:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T07:19:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T07:19:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T07:19:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T07:19:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T07:19:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T07:19:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T07:19:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T07:19:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T07:19:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T07:19:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T07:19:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T07:19:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T07:19:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T07:19:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T07:19:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T07:19:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T07:19:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T07:19:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T07:19:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T07:19:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T07:19:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T07:19:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T07:19:48.102 INFO:teuthology.orchestra.run.smithi189.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T07:19:48.102 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T07:19:48.102 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T07:19:48.102 INFO:teuthology.orchestra.run.smithi189.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T07:19:48.102 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T07:19:48.102 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T07:19:48.102 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T07:19:48.102 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T07:19:48.102 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T07:19:48.102 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T07:19:48.102 INFO:teuthology.orchestra.run.smithi189.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T07:19:48.102 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T07:19:48.102 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T07:19:48.102 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T07:19:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T07:19:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T07:19:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T07:19:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T07:19:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T07:19:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T07:19:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T07:19:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T07:19:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T07:19:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T07:19:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T07:19:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T07:19:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T07:19:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T07:19:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T07:19:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T07:19:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T07:19:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T07:19:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T07:19:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T07:19:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T07:19:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T07:19:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T07:19:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T07:19:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T07:19:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T07:19:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T07:19:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T07:19:48.105 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T07:19:48.105 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T07:19:48.105 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T07:19:48.105 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T07:19:48.105 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T07:19:48.105 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T07:19:48.105 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T07:19:48.105 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T07:19:48.105 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T07:19:48.105 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T07:19:48.105 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T07:19:48.105 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T07:19:48.105 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T07:19:48.105 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T07:19:48.106 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T07:19:48.106 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T07:19:48.106 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T07:19:48.106 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T07:19:48.106 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T07:19:48.106 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T07:19:48.106 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T07:19:48.106 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T07:19:48.106 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T07:19:48.106 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T07:19:48.106 INFO:teuthology.orchestra.run.smithi189.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T07:19:48.106 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T07:19:48.106 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T07:19:48.107 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T07:19:48.107 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T07:19:48.107 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T07:19:48.107 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T07:19:48.107 INFO:teuthology.orchestra.run.smithi189.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T07:19:48.107 INFO:teuthology.orchestra.run.smithi189.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T07:19:48.107 INFO:teuthology.orchestra.run.smithi189.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T07:19:48.107 INFO:teuthology.orchestra.run.smithi189.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T07:19:48.107 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T07:19:48.107 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T07:19:48.107 INFO:teuthology.orchestra.run.smithi189.stdout:Installing weak dependencies: 2024-09-19T07:19:48.107 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T07:19:48.107 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:19:48.108 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-19T07:19:48.108 INFO:teuthology.orchestra.run.smithi189.stdout:====================================================================================== 2024-09-19T07:19:48.108 INFO:teuthology.orchestra.run.smithi189.stdout:Install 148 Packages 2024-09-19T07:19:48.108 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrade 8 Packages 2024-09-19T07:19:48.108 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:19:48.108 INFO:teuthology.orchestra.run.smithi203.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 55 kB/s | 6.6 kB 00:00 2024-09-19T07:19:48.109 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 226 M 2024-09-19T07:19:48.109 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-19T07:19:48.291 INFO:teuthology.orchestra.run.smithi203.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 4.6 MB/s | 896 kB 00:00 2024-09-19T07:19:48.349 INFO:teuthology.orchestra.run.smithi203.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.4 MB/s | 145 kB 00:00 2024-09-19T07:19:48.567 INFO:teuthology.orchestra.run.smithi203.stdout:(4/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 11 MB/s | 2.4 MB 00:00 2024-09-19T07:19:48.633 INFO:teuthology.orchestra.run.smithi203.stdout:(5/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 8.3 MB/s | 5.4 MB 00:00 2024-09-19T07:19:48.742 INFO:teuthology.orchestra.run.smithi203.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 8.7 MB/s | 1.5 MB 00:00 2024-09-19T07:19:48.772 INFO:teuthology.orchestra.run.smithi189.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 45 kB/s | 6.6 kB 00:00 2024-09-19T07:19:48.942 INFO:teuthology.orchestra.run.smithi159.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 38 kB/s | 6.6 kB 00:00 2024-09-19T07:19:49.008 INFO:teuthology.orchestra.run.smithi203.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-09-19T07:19:49.097 INFO:teuthology.orchestra.run.smithi189.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 2.7 MB/s | 896 kB 00:00 2024-09-19T07:19:49.239 INFO:teuthology.orchestra.run.smithi189.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.0 MB/s | 145 kB 00:00 2024-09-19T07:19:49.268 INFO:teuthology.orchestra.run.smithi159.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 2.7 MB/s | 896 kB 00:00 2024-09-19T07:19:49.359 INFO:teuthology.orchestra.run.smithi159.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.6 MB/s | 145 kB 00:00 2024-09-19T07:19:49.801 INFO:teuthology.orchestra.run.smithi203.stdout:(8/156): ceph-radosgw-19.1.1-361.g2f599a46.el9. 13 MB/s | 11 MB 00:00 2024-09-19T07:19:49.935 INFO:teuthology.orchestra.run.smithi159.stdout:(4/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 4.2 MB/s | 2.4 MB 00:00 2024-09-19T07:19:49.947 INFO:teuthology.orchestra.run.smithi189.stdout:(4/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 3.4 MB/s | 2.4 MB 00:00 2024-09-19T07:19:50.010 INFO:teuthology.orchestra.run.smithi159.stdout:(5/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 4.4 MB/s | 5.4 MB 00:01 2024-09-19T07:19:50.035 INFO:teuthology.orchestra.run.smithi203.stdout:(9/156): ceph-common-19.1.1-361.g2f599a46.el9.x 10 MB/s | 21 MB 00:02 2024-09-19T07:19:50.060 INFO:teuthology.orchestra.run.smithi203.stdout:(10/156): ceph-selinux-19.1.1-361.g2f599a46.el9 97 kB/s | 25 kB 00:00 2024-09-19T07:19:50.181 INFO:teuthology.orchestra.run.smithi189.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 6.5 MB/s | 1.5 MB 00:00 2024-09-19T07:19:50.194 INFO:teuthology.orchestra.run.smithi159.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 5.9 MB/s | 1.5 MB 00:00 2024-09-19T07:19:50.235 INFO:teuthology.orchestra.run.smithi203.stdout:(11/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 17 MB 00:01 2024-09-19T07:19:50.248 INFO:teuthology.orchestra.run.smithi189.stdout:(6/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 3.3 MB/s | 5.4 MB 00:01 2024-09-19T07:19:50.260 INFO:teuthology.orchestra.run.smithi203.stdout:(12/156): cephfs-mirror-19.1.1-361.g2f599a46.el 1.1 MB/s | 228 kB 00:00 2024-09-19T07:19:50.311 INFO:teuthology.orchestra.run.smithi203.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 360 kB/s | 27 kB 00:00 2024-09-19T07:19:50.344 INFO:teuthology.orchestra.run.smithi203.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 379 kB/s | 32 kB 00:00 2024-09-19T07:19:50.445 INFO:teuthology.orchestra.run.smithi203.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 1.6 MB/s | 163 kB 00:00 2024-09-19T07:19:50.478 INFO:teuthology.orchestra.run.smithi203.stdout:(16/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 4.4 MB/s | 751 kB 00:00 2024-09-19T07:19:50.486 INFO:teuthology.orchestra.run.smithi159.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 9.9 MB/s | 4.7 MB 00:00 2024-09-19T07:19:50.512 INFO:teuthology.orchestra.run.smithi203.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 1.8 MB/s | 127 kB 00:00 2024-09-19T07:19:50.571 INFO:teuthology.orchestra.run.smithi203.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 5.4 MB/s | 506 kB 00:00 2024-09-19T07:19:50.663 INFO:teuthology.orchestra.run.smithi203.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 492 kB/s | 45 kB 00:00 2024-09-19T07:19:50.754 INFO:teuthology.orchestra.run.smithi203.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.4 MB/s | 136 kB 00:00 2024-09-19T07:19:50.855 INFO:teuthology.orchestra.run.smithi203.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.6 MB/s | 161 kB 00:00 2024-09-19T07:19:51.182 INFO:teuthology.orchestra.run.smithi189.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 4.7 MB/s | 4.7 MB 00:01 2024-09-19T07:19:51.597 INFO:teuthology.orchestra.run.smithi203.stdout:(22/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 4.9 MB/s | 5.3 MB 00:01 2024-09-19T07:19:51.622 INFO:teuthology.orchestra.run.smithi203.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 417 kB/s | 320 kB 00:00 2024-09-19T07:19:51.644 INFO:teuthology.orchestra.run.smithi159.stdout:(8/156): ceph-radosgw-19.1.1-361.g2f599a46.el9. 9.2 MB/s | 11 MB 00:01 2024-09-19T07:19:51.664 INFO:teuthology.orchestra.run.smithi203.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 4.4 MB/s | 300 kB 00:00 2024-09-19T07:19:51.689 INFO:teuthology.orchestra.run.smithi203.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.4 MB/s | 99 kB 00:00 2024-09-19T07:19:51.723 INFO:teuthology.orchestra.run.smithi203.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-19T07:19:51.798 INFO:teuthology.orchestra.run.smithi203.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.2 MB/s | 171 kB 00:00 2024-09-19T07:19:51.857 INFO:teuthology.orchestra.run.smithi203.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 543 kB/s | 32 kB 00:00 2024-09-19T07:19:51.903 INFO:teuthology.orchestra.run.smithi159.stdout:(9/156): ceph-common-19.1.1-361.g2f599a46.el9.x 6.8 MB/s | 21 MB 00:03 2024-09-19T07:19:51.928 INFO:teuthology.orchestra.run.smithi159.stdout:(10/156): ceph-selinux-19.1.1-361.g2f599a46.el9 89 kB/s | 25 kB 00:00 2024-09-19T07:19:51.932 INFO:teuthology.orchestra.run.smithi203.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.9 MB/s | 146 kB 00:00 2024-09-19T07:19:51.991 INFO:teuthology.orchestra.run.smithi203.stdout:(30/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 10 MB/s | 3.1 MB 00:00 2024-09-19T07:19:51.995 INFO:teuthology.orchestra.run.smithi159.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.3 MB/s | 228 kB 00:00 2024-09-19T07:19:52.070 INFO:teuthology.orchestra.run.smithi159.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 361 kB/s | 27 kB 00:00 2024-09-19T07:19:52.146 INFO:teuthology.orchestra.run.smithi159.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 423 kB/s | 32 kB 00:00 2024-09-19T07:19:52.157 INFO:teuthology.orchestra.run.smithi189.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 6.0 MB/s | 21 MB 00:03 2024-09-19T07:19:52.232 INFO:teuthology.orchestra.run.smithi189.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 337 kB/s | 25 kB 00:00 2024-09-19T07:19:52.337 INFO:teuthology.orchestra.run.smithi159.stdout:(14/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 7.9 MB/s | 17 MB 00:02 2024-09-19T07:19:52.366 INFO:teuthology.orchestra.run.smithi203.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 8.2 MB/s | 3.6 MB 00:00 2024-09-19T07:19:52.371 INFO:teuthology.orchestra.run.smithi159.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 3.3 MB/s | 751 kB 00:00 2024-09-19T07:19:52.413 INFO:teuthology.orchestra.run.smithi159.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.1 MB/s | 163 kB 00:00 2024-09-19T07:19:52.458 INFO:teuthology.orchestra.run.smithi203.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.7 MB/s | 249 kB 00:00 2024-09-19T07:19:52.480 INFO:teuthology.orchestra.run.smithi159.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 1.1 MB/s | 127 kB 00:00 2024-09-19T07:19:52.514 INFO:teuthology.orchestra.run.smithi159.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 4.9 MB/s | 506 kB 00:00 2024-09-19T07:19:52.534 INFO:teuthology.orchestra.run.smithi203.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 656 kB/s | 49 kB 00:00 2024-09-19T07:19:52.589 INFO:teuthology.orchestra.run.smithi159.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 604 kB/s | 45 kB 00:00 2024-09-19T07:19:52.626 INFO:teuthology.orchestra.run.smithi203.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 179 kB/s | 16 kB 00:00 2024-09-19T07:19:52.681 INFO:teuthology.orchestra.run.smithi159.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.5 MB/s | 136 kB 00:00 2024-09-19T07:19:52.726 INFO:teuthology.orchestra.run.smithi203.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 2.8 MB/s | 285 kB 00:00 2024-09-19T07:19:52.804 INFO:teuthology.orchestra.run.smithi159.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.3 MB/s | 161 kB 00:00 2024-09-19T07:19:52.818 INFO:teuthology.orchestra.run.smithi203.stdout:(36/156): ceph-mgr-diskprediction-local-19.1.1- 9.0 MB/s | 7.4 MB 00:00 2024-09-19T07:19:52.881 INFO:teuthology.orchestra.run.smithi159.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 4.0 MB/s | 320 kB 00:00 2024-09-19T07:19:52.902 INFO:teuthology.orchestra.run.smithi203.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 4.3 MB/s | 765 kB 00:00 2024-09-19T07:19:52.927 INFO:teuthology.orchestra.run.smithi203.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 168 kB/s | 18 kB 00:00 2024-09-19T07:19:52.956 INFO:teuthology.orchestra.run.smithi159.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 3.9 MB/s | 300 kB 00:00 2024-09-19T07:19:52.991 INFO:teuthology.orchestra.run.smithi189.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 6.1 MB/s | 17 MB 00:02 2024-09-19T07:19:53.032 INFO:teuthology.orchestra.run.smithi159.stdout:(24/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 9.7 MB/s | 5.3 MB 00:00 2024-09-19T07:19:53.041 INFO:teuthology.orchestra.run.smithi189.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 4.5 MB/s | 228 kB 00:00 2024-09-19T07:19:53.057 INFO:teuthology.orchestra.run.smithi159.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 990 kB/s | 99 kB 00:00 2024-09-19T07:19:53.077 INFO:teuthology.orchestra.run.smithi203.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 457 kB/s | 80 kB 00:00 2024-09-19T07:19:53.091 INFO:teuthology.orchestra.run.smithi159.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-19T07:19:53.100 INFO:teuthology.orchestra.run.smithi189.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 465 kB/s | 27 kB 00:00 2024-09-19T07:19:53.152 INFO:teuthology.orchestra.run.smithi203.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-19T07:19:53.166 INFO:teuthology.orchestra.run.smithi159.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.2 MB/s | 171 kB 00:00 2024-09-19T07:19:53.192 INFO:teuthology.orchestra.run.smithi189.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 346 kB/s | 32 kB 00:00 2024-09-19T07:19:53.194 INFO:teuthology.orchestra.run.smithi203.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.9 MB/s | 801 kB 00:00 2024-09-19T07:19:53.216 INFO:teuthology.orchestra.run.smithi159.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 639 kB/s | 32 kB 00:00 2024-09-19T07:19:53.245 INFO:teuthology.orchestra.run.smithi203.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-19T07:19:53.283 INFO:teuthology.orchestra.run.smithi159.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.1 MB/s | 146 kB 00:00 2024-09-19T07:19:53.295 INFO:teuthology.orchestra.run.smithi203.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-19T07:19:53.300 INFO:teuthology.orchestra.run.smithi189.stdout:(14/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 5.0 MB/s | 11 MB 00:02 2024-09-19T07:19:53.334 INFO:teuthology.orchestra.run.smithi189.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 5.2 MB/s | 751 kB 00:00 2024-09-19T07:19:53.337 INFO:teuthology.orchestra.run.smithi203.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-09-19T07:19:53.376 INFO:teuthology.orchestra.run.smithi159.stdout:(30/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 9.8 MB/s | 3.1 MB 00:00 2024-09-19T07:19:53.384 INFO:teuthology.orchestra.run.smithi189.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 1.9 MB/s | 163 kB 00:00 2024-09-19T07:19:53.409 INFO:teuthology.orchestra.run.smithi189.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 1.6 MB/s | 127 kB 00:00 2024-09-19T07:19:53.430 INFO:teuthology.orchestra.run.smithi203.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-09-19T07:19:53.488 INFO:teuthology.orchestra.run.smithi203.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-19T07:19:53.535 INFO:teuthology.orchestra.run.smithi189.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 3.3 MB/s | 506 kB 00:00 2024-09-19T07:19:53.546 INFO:teuthology.orchestra.run.smithi203.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 3.5 MB/s | 1.2 MB 00:00 2024-09-19T07:19:53.618 INFO:teuthology.orchestra.run.smithi189.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 543 kB/s | 45 kB 00:00 2024-09-19T07:19:53.626 INFO:teuthology.orchestra.run.smithi159.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 10 MB/s | 3.6 MB 00:00 2024-09-19T07:19:53.685 INFO:teuthology.orchestra.run.smithi189.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.0 MB/s | 136 kB 00:00 2024-09-19T07:19:53.709 INFO:teuthology.orchestra.run.smithi159.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.9 MB/s | 249 kB 00:00 2024-09-19T07:19:53.755 INFO:teuthology.orchestra.run.smithi203.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-09-19T07:19:53.760 INFO:teuthology.orchestra.run.smithi189.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.1 MB/s | 161 kB 00:00 2024-09-19T07:19:53.814 INFO:teuthology.orchestra.run.smithi203.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-09-19T07:19:53.818 INFO:teuthology.orchestra.run.smithi159.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 456 kB/s | 49 kB 00:00 2024-09-19T07:19:53.839 INFO:teuthology.orchestra.run.smithi203.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-09-19T07:19:53.856 INFO:teuthology.orchestra.run.smithi203.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 518 kB/s | 22 kB 00:00 2024-09-19T07:19:53.885 INFO:teuthology.orchestra.run.smithi159.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 247 kB/s | 16 kB 00:00 2024-09-19T07:19:53.886 INFO:teuthology.orchestra.run.smithi189.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 2.5 MB/s | 320 kB 00:00 2024-09-19T07:19:53.898 INFO:teuthology.orchestra.run.smithi203.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-19T07:19:53.916 INFO:teuthology.orchestra.run.smithi203.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 554 kB/s | 33 kB 00:00 2024-09-19T07:19:53.966 INFO:teuthology.orchestra.run.smithi203.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-19T07:19:53.977 INFO:teuthology.orchestra.run.smithi159.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 3.1 MB/s | 285 kB 00:00 2024-09-19T07:19:54.014 INFO:teuthology.orchestra.run.smithi203.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 231 kB/s | 11 kB 00:00 2024-09-19T07:19:54.028 INFO:teuthology.orchestra.run.smithi189.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 2.1 MB/s | 300 kB 00:00 2024-09-19T07:19:54.064 INFO:teuthology.orchestra.run.smithi203.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-09-19T07:19:54.069 INFO:teuthology.orchestra.run.smithi159.stdout:(36/156): cephadm-19.1.1-361.g2f599a46.el9.noar 8.2 MB/s | 765 kB 00:00 2024-09-19T07:19:54.090 INFO:teuthology.orchestra.run.smithi203.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-19T07:19:54.115 INFO:teuthology.orchestra.run.smithi203.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-09-19T07:19:54.119 INFO:teuthology.orchestra.run.smithi189.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.1 MB/s | 99 kB 00:00 2024-09-19T07:19:54.149 INFO:teuthology.orchestra.run.smithi203.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-19T07:19:54.178 INFO:teuthology.orchestra.run.smithi159.stdout:(37/156): ceph-mgr-diskprediction-local-19.1.1- 9.2 MB/s | 7.4 MB 00:00 2024-09-19T07:19:54.191 INFO:teuthology.orchestra.run.smithi203.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-09-19T07:19:54.194 INFO:teuthology.orchestra.run.smithi159.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 146 kB/s | 18 kB 00:00 2024-09-19T07:19:54.203 INFO:teuthology.orchestra.run.smithi189.stdout:(25/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 6.7 MB/s | 5.3 MB 00:00 2024-09-19T07:19:54.208 INFO:teuthology.orchestra.run.smithi203.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 417 kB/s | 24 kB 00:00 2024-09-19T07:19:54.228 INFO:teuthology.orchestra.run.smithi189.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 790 kB/s | 85 kB 00:00 2024-09-19T07:19:54.233 INFO:teuthology.orchestra.run.smithi203.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-19T07:19:54.267 INFO:teuthology.orchestra.run.smithi203.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-19T07:19:54.292 INFO:teuthology.orchestra.run.smithi203.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-19T07:19:54.295 INFO:teuthology.orchestra.run.smithi189.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-19T07:19:54.309 INFO:teuthology.orchestra.run.smithi203.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 916 kB/s | 38 kB 00:00 2024-09-19T07:19:54.353 INFO:teuthology.orchestra.run.smithi189.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 546 kB/s | 32 kB 00:00 2024-09-19T07:19:54.421 INFO:teuthology.orchestra.run.smithi189.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.1 MB/s | 146 kB 00:00 2024-09-19T07:19:54.460 INFO:teuthology.orchestra.run.smithi203.stdout:(66/156): openblas-openmp-0.3.26-2.el9.x86_64.r 30 MB/s | 4.9 MB 00:00 2024-09-19T07:19:54.526 INFO:teuthology.orchestra.run.smithi203.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-19T07:19:54.568 INFO:teuthology.orchestra.run.smithi203.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-19T07:19:54.587 INFO:teuthology.orchestra.run.smithi189.stdout:(30/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 8.1 MB/s | 3.1 MB 00:00 2024-09-19T07:19:54.602 INFO:teuthology.orchestra.run.smithi203.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-19T07:19:54.627 INFO:teuthology.orchestra.run.smithi203.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-19T07:19:54.703 INFO:teuthology.orchestra.run.smithi203.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-19T07:19:54.761 INFO:teuthology.orchestra.run.smithi203.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-19T07:19:54.837 INFO:teuthology.orchestra.run.smithi159.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 121 kB/s | 80 kB 00:00 2024-09-19T07:19:54.845 INFO:teuthology.orchestra.run.smithi203.stdout:(73/156): postgresql-server-13.16-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-09-19T07:19:54.895 INFO:teuthology.orchestra.run.smithi203.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-19T07:19:55.004 INFO:teuthology.orchestra.run.smithi203.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 6.0 MB 00:00 2024-09-19T07:19:55.005 INFO:teuthology.orchestra.run.smithi189.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 6.1 MB/s | 3.6 MB 00:00 2024-09-19T07:19:55.029 INFO:teuthology.orchestra.run.smithi203.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-19T07:19:55.055 INFO:teuthology.orchestra.run.smithi189.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 4.9 MB/s | 249 kB 00:00 2024-09-19T07:19:55.063 INFO:teuthology.orchestra.run.smithi203.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-19T07:19:55.088 INFO:teuthology.orchestra.run.smithi203.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 435 kB/s | 26 kB 00:00 2024-09-19T07:19:55.096 INFO:teuthology.orchestra.run.smithi189.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 1.2 MB/s | 49 kB 00:00 2024-09-19T07:19:55.113 INFO:teuthology.orchestra.run.smithi203.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-19T07:19:55.138 INFO:teuthology.orchestra.run.smithi189.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 394 kB/s | 16 kB 00:00 2024-09-19T07:19:55.139 INFO:teuthology.orchestra.run.smithi203.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-19T07:19:55.164 INFO:teuthology.orchestra.run.smithi203.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 689 kB/s | 35 kB 00:00 2024-09-19T07:19:55.197 INFO:teuthology.orchestra.run.smithi189.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 4.8 MB/s | 285 kB 00:00 2024-09-19T07:19:55.223 INFO:teuthology.orchestra.run.smithi203.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-19T07:19:55.281 INFO:teuthology.orchestra.run.smithi203.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-19T07:19:55.289 INFO:teuthology.orchestra.run.smithi189.stdout:(36/156): ceph-mgr-diskprediction-local-19.1.1- 11 MB/s | 7.4 MB 00:00 2024-09-19T07:19:55.339 INFO:teuthology.orchestra.run.smithi189.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 5.3 MB/s | 765 kB 00:00 2024-09-19T07:19:55.357 INFO:teuthology.orchestra.run.smithi203.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-09-19T07:19:55.364 INFO:teuthology.orchestra.run.smithi189.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 243 kB/s | 18 kB 00:00 2024-09-19T07:19:55.382 INFO:teuthology.orchestra.run.smithi203.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 769 kB/s | 77 kB 00:00 2024-09-19T07:19:55.387 INFO:teuthology.orchestra.run.smithi159.stdout:(40/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 14 MB/s | 49 MB 00:03 2024-09-19T07:19:55.399 INFO:teuthology.orchestra.run.smithi203.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-19T07:19:55.441 INFO:teuthology.orchestra.run.smithi203.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-19T07:19:55.466 INFO:teuthology.orchestra.run.smithi203.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-19T07:19:55.481 INFO:teuthology.orchestra.run.smithi189.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 563 kB/s | 80 kB 00:00 2024-09-19T07:19:55.509 INFO:teuthology.orchestra.run.smithi203.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-09-19T07:19:55.512 INFO:teuthology.orchestra.run.smithi159.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 608 kB/s | 801 kB 00:01 2024-09-19T07:19:55.525 INFO:teuthology.orchestra.run.smithi203.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 865 kB/s | 51 kB 00:00 2024-09-19T07:19:55.551 INFO:teuthology.orchestra.run.smithi203.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 54 kB 00:00 2024-09-19T07:19:55.573 INFO:teuthology.orchestra.run.smithi189.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-19T07:19:55.593 INFO:teuthology.orchestra.run.smithi203.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 996 kB/s | 42 kB 00:00 2024-09-19T07:19:55.632 INFO:teuthology.orchestra.run.smithi189.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.9 MB/s | 801 kB 00:00 2024-09-19T07:19:55.643 INFO:teuthology.orchestra.run.smithi203.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-09-19T07:19:55.657 INFO:teuthology.orchestra.run.smithi189.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.9 MB/s | 253 kB 00:00 2024-09-19T07:19:55.702 INFO:teuthology.orchestra.run.smithi203.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-19T07:19:55.724 INFO:teuthology.orchestra.run.smithi189.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-19T07:19:55.752 INFO:teuthology.orchestra.run.smithi203.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-09-19T07:19:55.971 INFO:teuthology.orchestra.run.smithi159.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 169 kB/s | 191 kB 00:01 2024-09-19T07:19:56.146 INFO:teuthology.orchestra.run.smithi159.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 334 kB/s | 253 kB 00:00 2024-09-19T07:19:56.200 INFO:teuthology.orchestra.run.smithi189.stdout:(44/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 12 MB/s | 49 MB 00:03 2024-09-19T07:19:56.225 INFO:teuthology.orchestra.run.smithi189.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 270 kB/s | 135 kB 00:00 2024-09-19T07:19:56.228 INFO:teuthology.orchestra.run.smithi203.stdout:(96/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 7.9 MB/s | 49 MB 00:06 2024-09-19T07:19:56.245 INFO:teuthology.orchestra.run.smithi203.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 96 kB/s | 47 kB 00:00 2024-09-19T07:19:56.259 INFO:teuthology.orchestra.run.smithi189.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 2.0 MB/s | 1.2 MB 00:00 2024-09-19T07:19:56.286 INFO:teuthology.orchestra.run.smithi203.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-19T07:19:56.350 INFO:teuthology.orchestra.run.smithi189.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-09-19T07:19:56.401 INFO:teuthology.orchestra.run.smithi189.stdout:(48/156): python3-requests-2.25.1-8.el9.noarch. 623 kB/s | 125 kB 00:00 2024-09-19T07:19:56.430 INFO:teuthology.orchestra.run.smithi159.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-19T07:19:56.479 INFO:teuthology.orchestra.run.smithi203.stdout:(99/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-09-19T07:19:56.504 INFO:teuthology.orchestra.run.smithi203.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 442 kB/s | 96 kB 00:00 2024-09-19T07:19:56.529 INFO:teuthology.orchestra.run.smithi203.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T07:19:56.568 INFO:teuthology.orchestra.run.smithi189.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 641 kB/s | 107 kB 00:00 2024-09-19T07:19:56.588 INFO:teuthology.orchestra.run.smithi203.stdout:(102/156): socat-1.7.4.1-6.el9.x86_64.rpm 846 kB/s | 304 kB 00:00 2024-09-19T07:19:56.593 INFO:teuthology.orchestra.run.smithi189.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 689 kB 00:00 2024-09-19T07:19:56.613 INFO:teuthology.orchestra.run.smithi203.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-09-19T07:19:56.618 INFO:teuthology.orchestra.run.smithi189.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-09-19T07:19:56.663 INFO:teuthology.orchestra.run.smithi159.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 154 kB/s | 106 kB 00:00 2024-09-19T07:19:56.669 INFO:teuthology.orchestra.run.smithi189.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-09-19T07:19:56.672 INFO:teuthology.orchestra.run.smithi203.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-19T07:19:56.694 INFO:teuthology.orchestra.run.smithi189.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-09-19T07:19:56.697 INFO:teuthology.orchestra.run.smithi203.stdout:(105/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 232 kB/s | 45 kB 00:00 2024-09-19T07:19:56.728 INFO:teuthology.orchestra.run.smithi189.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-09-19T07:19:56.731 INFO:teuthology.orchestra.run.smithi203.stdout:(106/156): gperftools-libs-2.9.1-3.el9.x86_64.r 5.1 MB/s | 308 kB 00:00 2024-09-19T07:19:56.747 INFO:teuthology.orchestra.run.smithi203.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 386 kB/s | 19 kB 00:00 2024-09-19T07:19:56.753 INFO:teuthology.orchestra.run.smithi189.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-09-19T07:19:56.765 INFO:teuthology.orchestra.run.smithi203.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-19T07:19:56.778 INFO:teuthology.orchestra.run.smithi189.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-19T07:19:56.790 INFO:teuthology.orchestra.run.smithi203.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T07:19:56.803 INFO:teuthology.orchestra.run.smithi189.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-19T07:19:56.823 INFO:teuthology.orchestra.run.smithi203.stdout:(110/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.6 MB/s | 863 kB 00:00 2024-09-19T07:19:56.829 INFO:teuthology.orchestra.run.smithi189.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-09-19T07:19:56.849 INFO:teuthology.orchestra.run.smithi203.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-19T07:19:56.907 INFO:teuthology.orchestra.run.smithi203.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-09-19T07:19:56.921 INFO:teuthology.orchestra.run.smithi189.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 6.2 MB/s | 581 kB 00:00 2024-09-19T07:19:56.927 INFO:teuthology.orchestra.run.smithi159.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 173 kB/s | 135 kB 00:00 2024-09-19T07:19:56.946 INFO:teuthology.orchestra.run.smithi189.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-09-19T07:19:56.971 INFO:teuthology.orchestra.run.smithi189.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 488 kB/s | 24 kB 00:00 2024-09-19T07:19:56.996 INFO:teuthology.orchestra.run.smithi189.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-19T07:19:56.999 INFO:teuthology.orchestra.run.smithi203.stdout:(113/156): libarrow-9.0.0-11.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-09-19T07:19:57.025 INFO:teuthology.orchestra.run.smithi203.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T07:19:57.050 INFO:teuthology.orchestra.run.smithi203.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.2 MB/s | 32 kB 00:00 2024-09-19T07:19:57.055 INFO:teuthology.orchestra.run.smithi189.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-19T07:19:57.076 INFO:teuthology.orchestra.run.smithi203.stdout:(116/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.7 MB/s | 839 kB 00:00 2024-09-19T07:19:57.105 INFO:teuthology.orchestra.run.smithi189.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-19T07:19:57.129 INFO:teuthology.orchestra.run.smithi203.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 181 kB/s | 14 kB 00:00 2024-09-19T07:19:57.139 INFO:teuthology.orchestra.run.smithi189.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-09-19T07:19:57.147 INFO:teuthology.orchestra.run.smithi159.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 174 kB/s | 125 kB 00:00 2024-09-19T07:19:57.160 INFO:teuthology.orchestra.run.smithi203.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-09-19T07:19:57.185 INFO:teuthology.orchestra.run.smithi203.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-09-19T07:19:57.218 INFO:teuthology.orchestra.run.smithi203.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 3.9 MB/s | 358 kB 00:00 2024-09-19T07:19:57.244 INFO:teuthology.orchestra.run.smithi203.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-09-19T07:19:57.261 INFO:teuthology.orchestra.run.smithi203.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.9 MB/s | 220 kB 00:00 2024-09-19T07:19:57.286 INFO:teuthology.orchestra.run.smithi203.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-19T07:19:57.311 INFO:teuthology.orchestra.run.smithi203.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-09-19T07:19:57.337 INFO:teuthology.orchestra.run.smithi203.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-19T07:19:57.362 INFO:teuthology.orchestra.run.smithi203.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-19T07:19:57.387 INFO:teuthology.orchestra.run.smithi203.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-19T07:19:57.413 INFO:teuthology.orchestra.run.smithi203.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-19T07:19:57.438 INFO:teuthology.orchestra.run.smithi203.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T07:19:57.463 INFO:teuthology.orchestra.run.smithi203.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T07:19:57.464 INFO:teuthology.orchestra.run.smithi159.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 272 kB/s | 215 kB 00:00 2024-09-19T07:19:57.489 INFO:teuthology.orchestra.run.smithi203.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T07:19:57.522 INFO:teuthology.orchestra.run.smithi203.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-19T07:19:57.581 INFO:teuthology.orchestra.run.smithi203.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 282 kB/s | 16 kB 00:00 2024-09-19T07:19:57.614 INFO:teuthology.orchestra.run.smithi203.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-09-19T07:19:57.639 INFO:teuthology.orchestra.run.smithi189.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.1 MB/s | 3.0 MB 00:00 2024-09-19T07:19:57.639 INFO:teuthology.orchestra.run.smithi203.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-19T07:19:57.656 INFO:teuthology.orchestra.run.smithi203.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 675 kB/s | 28 kB 00:00 2024-09-19T07:19:57.673 INFO:teuthology.orchestra.run.smithi203.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 917 kB/s | 31 kB 00:00 2024-09-19T07:19:57.690 INFO:teuthology.orchestra.run.smithi203.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-19T07:19:57.716 INFO:teuthology.orchestra.run.smithi203.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-09-19T07:19:57.733 INFO:teuthology.orchestra.run.smithi203.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 853 kB/s | 36 kB 00:00 2024-09-19T07:19:57.758 INFO:teuthology.orchestra.run.smithi203.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-19T07:19:57.800 INFO:teuthology.orchestra.run.smithi203.stdout:(142/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.7 MB/s | 2.0 MB 00:00 2024-09-19T07:19:57.814 INFO:teuthology.orchestra.run.smithi159.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 787 kB/s | 689 kB 00:00 2024-09-19T07:19:57.826 INFO:teuthology.orchestra.run.smithi203.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-19T07:19:57.851 INFO:teuthology.orchestra.run.smithi203.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-19T07:19:57.873 INFO:teuthology.orchestra.run.smithi189.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 6.4 MB/s | 4.9 MB 00:00 2024-09-19T07:19:57.876 INFO:teuthology.orchestra.run.smithi203.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-19T07:19:57.902 INFO:teuthology.orchestra.run.smithi203.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T07:19:57.927 INFO:teuthology.orchestra.run.smithi203.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-19T07:19:57.977 INFO:teuthology.orchestra.run.smithi203.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-19T07:19:57.981 INFO:teuthology.orchestra.run.smithi189.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-09-19T07:19:58.006 INFO:teuthology.orchestra.run.smithi189.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 6.7 MB/s | 910 kB 00:00 2024-09-19T07:19:58.040 INFO:teuthology.orchestra.run.smithi189.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-19T07:19:58.115 INFO:teuthology.orchestra.run.smithi189.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-19T07:19:58.131 INFO:teuthology.orchestra.run.smithi159.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 959 kB/s | 943 kB 00:00 2024-09-19T07:19:58.156 INFO:teuthology.orchestra.run.smithi159.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 154 kB/s | 107 kB 00:00 2024-09-19T07:19:58.174 INFO:teuthology.orchestra.run.smithi189.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-19T07:19:58.219 INFO:teuthology.orchestra.run.smithi203.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-19T07:19:58.278 INFO:teuthology.orchestra.run.smithi203.stdout:(150/156): librbd1-19.1.1-361.g2f599a46.el9.x86 9.0 MB/s | 3.2 MB 00:00 2024-09-19T07:19:58.349 INFO:teuthology.orchestra.run.smithi189.stdout:(73/156): python3-babel-2.9.1-2.el9.noarch.rpm 34 MB/s | 6.0 MB 00:00 2024-09-19T07:19:58.408 INFO:teuthology.orchestra.run.smithi189.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-19T07:19:58.420 INFO:teuthology.orchestra.run.smithi203.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 289 kB/s | 41 kB 00:00 2024-09-19T07:19:58.423 INFO:teuthology.orchestra.run.smithi159.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 36 kB/s | 22 kB 00:00 2024-09-19T07:19:58.467 INFO:teuthology.orchestra.run.smithi189.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-19T07:19:58.525 INFO:teuthology.orchestra.run.smithi189.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-09-19T07:19:58.537 INFO:teuthology.orchestra.run.smithi203.stdout:(152/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.1 MB/s | 249 kB 00:00 2024-09-19T07:19:58.584 INFO:teuthology.orchestra.run.smithi189.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 439 kB/s | 26 kB 00:00 2024-09-19T07:19:58.612 INFO:teuthology.orchestra.run.smithi203.stdout:(153/156): NetworkManager-libnm-1.51.0-1.el9.x8 4.6 MB/s | 1.8 MB 00:00 2024-09-19T07:19:58.684 INFO:teuthology.orchestra.run.smithi189.stdout:(78/156): postgresql-server-13.16-1.el9.x86_64. 8.5 MB/s | 5.8 MB 00:00 2024-09-19T07:19:58.707 INFO:teuthology.orchestra.run.smithi159.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-09-19T07:19:58.709 INFO:teuthology.orchestra.run.smithi189.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-09-19T07:19:58.737 INFO:teuthology.orchestra.run.smithi203.stdout:(154/156): NetworkManager-1.51.0-1.el9.x86_64.r 3.1 MB/s | 2.3 MB 00:00 2024-09-19T07:19:58.743 INFO:teuthology.orchestra.run.smithi189.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-19T07:19:58.768 INFO:teuthology.orchestra.run.smithi189.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 592 kB/s | 35 kB 00:00 2024-09-19T07:19:58.779 INFO:teuthology.orchestra.run.smithi203.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 5.6 MB/s | 1.4 MB 00:00 2024-09-19T07:19:58.835 INFO:teuthology.orchestra.run.smithi189.stdout:(82/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.6 MB/s | 4.4 MB 00:01 2024-09-19T07:19:58.860 INFO:teuthology.orchestra.run.smithi189.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-09-19T07:19:58.919 INFO:teuthology.orchestra.run.smithi189.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-19T07:19:58.946 INFO:teuthology.orchestra.run.smithi203.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 7.2 MB/s | 2.4 MB 00:00 2024-09-19T07:19:58.952 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-09-19T07:19:58.953 INFO:teuthology.orchestra.run.smithi203.stdout:Total 20 MB/s | 226 MB 00:11 2024-09-19T07:19:58.978 INFO:teuthology.orchestra.run.smithi189.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 723 kB/s | 42 kB 00:00 2024-09-19T07:19:58.991 INFO:teuthology.orchestra.run.smithi159.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 376 kB/s | 313 kB 00:00 2024-09-19T07:19:59.003 INFO:teuthology.orchestra.run.smithi189.stdout:(86/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 222 kB 00:00 2024-09-19T07:19:59.045 INFO:teuthology.orchestra.run.smithi189.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-09-19T07:19:59.103 INFO:teuthology.orchestra.run.smithi189.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-19T07:19:59.129 INFO:teuthology.orchestra.run.smithi189.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-19T07:19:59.162 INFO:teuthology.orchestra.run.smithi189.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 869 kB/s | 51 kB 00:00 2024-09-19T07:19:59.188 INFO:teuthology.orchestra.run.smithi189.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 917 kB/s | 54 kB 00:00 2024-09-19T07:19:59.199 INFO:teuthology.orchestra.run.smithi159.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 37 kB/s | 18 kB 00:00 2024-09-19T07:19:59.254 INFO:teuthology.orchestra.run.smithi189.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 627 kB/s | 42 kB 00:00 2024-09-19T07:19:59.355 INFO:teuthology.orchestra.run.smithi189.stdout:(93/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.3 MB/s | 5.1 MB 00:00 2024-09-19T07:19:59.380 INFO:teuthology.orchestra.run.smithi189.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 352 kB/s | 44 kB 00:00 2024-09-19T07:19:59.413 INFO:teuthology.orchestra.run.smithi189.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-19T07:19:59.439 INFO:teuthology.orchestra.run.smithi189.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-09-19T07:19:59.472 INFO:teuthology.orchestra.run.smithi189.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-09-19T07:19:59.475 INFO:teuthology.orchestra.run.smithi159.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 23 kB/s | 11 kB 00:00 2024-09-19T07:19:59.531 INFO:teuthology.orchestra.run.smithi189.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 990 kB/s | 58 kB 00:00 2024-09-19T07:19:59.600 INFO:teuthology.orchestra.run.smithi159.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.5 MB/s | 3.0 MB 00:01 2024-09-19T07:19:59.690 INFO:teuthology.orchestra.run.smithi189.stdout:(99/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-09-19T07:19:59.715 INFO:teuthology.orchestra.run.smithi189.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 522 kB/s | 96 kB 00:00 2024-09-19T07:19:59.740 INFO:teuthology.orchestra.run.smithi189.stdout:(101/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.0 MB/s | 304 kB 00:00 2024-09-19T07:19:59.757 INFO:teuthology.orchestra.run.smithi189.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 952 kB/s | 64 kB 00:00 2024-09-19T07:19:59.774 INFO:teuthology.orchestra.run.smithi189.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 757 kB/s | 45 kB 00:00 2024-09-19T07:19:59.833 INFO:teuthology.orchestra.run.smithi189.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-19T07:19:59.858 INFO:teuthology.orchestra.run.smithi189.stdout:(105/156): lua-devel-5.4.4-4.el9.x86_64.rpm 189 kB/s | 22 kB 00:00 2024-09-19T07:19:59.892 INFO:teuthology.orchestra.run.smithi189.stdout:(106/156): gperftools-libs-2.9.1-3.el9.x86_64.r 5.1 MB/s | 308 kB 00:00 2024-09-19T07:19:59.925 INFO:teuthology.orchestra.run.smithi189.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-09-19T07:19:59.951 INFO:teuthology.orchestra.run.smithi189.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-19T07:19:59.975 INFO:teuthology.orchestra.run.smithi159.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 207 kB/s | 160 kB 00:00 2024-09-19T07:19:59.984 INFO:teuthology.orchestra.run.smithi189.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-19T07:20:00.017 INFO:teuthology.orchestra.run.smithi189.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-19T07:20:00.043 INFO:teuthology.orchestra.run.smithi189.stdout:(111/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.0 MB/s | 863 kB 00:00 2024-09-19T07:20:00.093 INFO:teuthology.orchestra.run.smithi189.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-19T07:20:00.141 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-09-19T07:20:00.159 INFO:teuthology.orchestra.run.smithi159.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 66 kB/s | 45 kB 00:00 2024-09-19T07:20:00.177 INFO:teuthology.orchestra.run.smithi189.stdout:(113/156): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-19T07:20:00.202 INFO:teuthology.orchestra.run.smithi189.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T07:20:00.227 INFO:teuthology.orchestra.run.smithi189.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T07:20:00.253 INFO:teuthology.orchestra.run.smithi189.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-19T07:20:00.278 INFO:teuthology.orchestra.run.smithi189.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-09-19T07:20:00.294 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-09-19T07:20:00.294 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-09-19T07:20:00.303 INFO:teuthology.orchestra.run.smithi189.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-19T07:20:00.329 INFO:teuthology.orchestra.run.smithi189.stdout:(119/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.9 MB/s | 839 kB 00:00 2024-09-19T07:20:00.362 INFO:teuthology.orchestra.run.smithi189.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-19T07:20:00.387 INFO:teuthology.orchestra.run.smithi189.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-09-19T07:20:00.421 INFO:teuthology.orchestra.run.smithi189.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.3 MB/s | 220 kB 00:00 2024-09-19T07:20:00.446 INFO:teuthology.orchestra.run.smithi189.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-09-19T07:20:00.468 INFO:teuthology.orchestra.run.smithi159.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 764 kB/s | 662 kB 00:00 2024-09-19T07:20:00.471 INFO:teuthology.orchestra.run.smithi189.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-09-19T07:20:00.497 INFO:teuthology.orchestra.run.smithi189.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-19T07:20:00.522 INFO:teuthology.orchestra.run.smithi189.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-09-19T07:20:00.547 INFO:teuthology.orchestra.run.smithi189.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-19T07:20:00.573 INFO:teuthology.orchestra.run.smithi189.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-19T07:20:00.606 INFO:teuthology.orchestra.run.smithi189.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-19T07:20:00.632 INFO:teuthology.orchestra.run.smithi189.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T07:20:00.665 INFO:teuthology.orchestra.run.smithi189.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-19T07:20:00.690 INFO:teuthology.orchestra.run.smithi189.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 989 kB/s | 58 kB 00:00 2024-09-19T07:20:00.715 INFO:teuthology.orchestra.run.smithi189.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 657 kB/s | 16 kB 00:00 2024-09-19T07:20:00.718 INFO:teuthology.orchestra.run.smithi159.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 44 kB/s | 24 kB 00:00 2024-09-19T07:20:00.741 INFO:teuthology.orchestra.run.smithi189.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-19T07:20:00.767 INFO:teuthology.orchestra.run.smithi189.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-19T07:20:00.791 INFO:teuthology.orchestra.run.smithi189.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-19T07:20:00.817 INFO:teuthology.orchestra.run.smithi189.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-19T07:20:00.835 INFO:teuthology.orchestra.run.smithi159.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 676 kB/s | 581 kB 00:00 2024-09-19T07:20:00.842 INFO:teuthology.orchestra.run.smithi189.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T07:20:00.867 INFO:teuthology.orchestra.run.smithi189.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-19T07:20:00.884 INFO:teuthology.orchestra.run.smithi189.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 851 kB/s | 36 kB 00:00 2024-09-19T07:20:00.943 INFO:teuthology.orchestra.run.smithi189.stdout:(141/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.5 MB/s | 2.0 MB 00:00 2024-09-19T07:20:00.969 INFO:teuthology.orchestra.run.smithi189.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 739 kB/s | 74 kB 00:00 2024-09-19T07:20:00.994 INFO:teuthology.orchestra.run.smithi189.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T07:20:01.019 INFO:teuthology.orchestra.run.smithi189.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-19T07:20:01.044 INFO:teuthology.orchestra.run.smithi189.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-19T07:20:01.069 INFO:teuthology.orchestra.run.smithi189.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T07:20:01.095 INFO:teuthology.orchestra.run.smithi189.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-19T07:20:01.127 INFO:teuthology.orchestra.run.smithi159.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 72 kB/s | 47 kB 00:00 2024-09-19T07:20:01.145 INFO:teuthology.orchestra.run.smithi189.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-19T07:20:01.295 INFO:teuthology.orchestra.run.smithi189.stdout:(149/156): NetworkManager-1.51.0-1.el9.x86_64.r 16 MB/s | 2.3 MB 00:00 2024-09-19T07:20:01.370 INFO:teuthology.orchestra.run.smithi189.stdout:(150/156): librados2-19.1.1-361.g2f599a46.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-19T07:20:01.413 INFO:teuthology.orchestra.run.smithi189.stdout:(151/156): NetworkManager-libnm-1.51.0-1.el9.x8 15 MB/s | 1.8 MB 00:00 2024-09-19T07:20:01.479 INFO:teuthology.orchestra.run.smithi189.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 376 kB/s | 41 kB 00:00 2024-09-19T07:20:01.505 INFO:teuthology.orchestra.run.smithi189.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.6 MB/s | 249 kB 00:00 2024-09-19T07:20:01.565 INFO:teuthology.orchestra.run.smithi189.stdout:(154/156): librbd1-19.1.1-361.g2f599a46.el9.x86 6.7 MB/s | 3.2 MB 00:00 2024-09-19T07:20:01.568 INFO:teuthology.orchestra.run.smithi159.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 257 kB/s | 188 kB 00:00 2024-09-19T07:20:01.594 INFO:teuthology.orchestra.run.smithi159.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 338 kB/s | 292 kB 00:00 2024-09-19T07:20:01.642 INFO:teuthology.orchestra.run.smithi189.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-09-19T07:20:01.694 INFO:teuthology.orchestra.run.smithi159.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 68 kB/s | 38 kB 00:00 2024-09-19T07:20:01.797 INFO:teuthology.orchestra.run.smithi189.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 4.3 MB/s | 1.4 MB 00:00 2024-09-19T07:20:01.802 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-19T07:20:01.802 INFO:teuthology.orchestra.run.smithi189.stdout:Total 17 MB/s | 226 MB 00:13 2024-09-19T07:20:02.114 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-09-19T07:20:02.115 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-09-19T07:20:02.669 INFO:teuthology.orchestra.run.smithi159.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.5 MB/s | 4.9 MB 00:01 2024-09-19T07:20:02.711 INFO:teuthology.orchestra.run.smithi159.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:01 2024-09-19T07:20:02.796 INFO:teuthology.orchestra.run.smithi159.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-09-19T07:20:03.042 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-19T07:20:03.195 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-19T07:20:03.195 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-19T07:20:03.412 INFO:teuthology.orchestra.run.smithi159.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 196 kB/s | 137 kB 00:00 2024-09-19T07:20:03.654 INFO:teuthology.orchestra.run.smithi159.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 925 kB/s | 910 kB 00:00 2024-09-19T07:20:03.987 INFO:teuthology.orchestra.run.smithi159.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 4.8 MB/s | 5.8 MB 00:01 2024-09-19T07:20:04.329 INFO:teuthology.orchestra.run.smithi159.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 132 kB/s | 87 kB 00:00 2024-09-19T07:20:04.438 INFO:teuthology.orchestra.run.smithi159.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:01 2024-09-19T07:20:05.007 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-19T07:20:05.008 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-19T07:20:05.013 INFO:teuthology.orchestra.run.smithi159.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 90 kB/s | 61 kB 00:00 2024-09-19T07:20:05.522 INFO:teuthology.orchestra.run.smithi159.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 226 kB/s | 245 kB 00:01 2024-09-19T07:20:05.797 INFO:teuthology.orchestra.run.smithi159.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.3 MB/s | 6.0 MB 00:01 2024-09-19T07:20:05.847 INFO:teuthology.orchestra.run.smithi159.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 298 kB/s | 248 kB 00:00 2024-09-19T07:20:06.131 INFO:teuthology.orchestra.run.smithi159.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 42 kB/s | 26 kB 00:00 2024-09-19T07:20:06.298 INFO:teuthology.orchestra.run.smithi159.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 39 kB/s | 19 kB 00:00 2024-09-19T07:20:06.606 INFO:teuthology.orchestra.run.smithi159.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 227 kB/s | 172 kB 00:00 2024-09-19T07:20:06.690 INFO:teuthology.orchestra.run.smithi159.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 62 kB/s | 35 kB 00:00 2024-09-19T07:20:07.476 INFO:teuthology.orchestra.run.smithi159.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.3 MB/s | 5.1 MB 00:01 2024-09-19T07:20:07.516 INFO:teuthology.orchestra.run.smithi159.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 276 kB/s | 250 kB 00:00 2024-09-19T07:20:07.541 INFO:teuthology.orchestra.run.smithi159.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 261 kB/s | 222 kB 00:00 2024-09-19T07:20:08.116 INFO:teuthology.orchestra.run.smithi159.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 121 kB/s | 77 kB 00:00 2024-09-19T07:20:08.175 INFO:teuthology.orchestra.run.smithi159.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 64 kB/s | 42 kB 00:00 2024-09-19T07:20:08.378 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-09-19T07:20:08.442 INFO:teuthology.orchestra.run.smithi159.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 299 kB/s | 269 kB 00:00 2024-09-19T07:20:08.483 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T07:20:08.606 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T07:20:08.804 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T07:20:08.843 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T07:20:08.859 INFO:teuthology.orchestra.run.smithi159.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 214 kB/s | 159 kB 00:00 2024-09-19T07:20:08.868 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T07:20:08.913 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T07:20:09.059 INFO:teuthology.orchestra.run.smithi159.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 316 kB/s | 279 kB 00:00 2024-09-19T07:20:09.101 INFO:teuthology.orchestra.run.smithi159.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 77 kB/s | 51 kB 00:00 2024-09-19T07:20:09.250 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T07:20:09.376 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:20:09.493 INFO:teuthology.orchestra.run.smithi159.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 85 kB/s | 54 kB 00:00 2024-09-19T07:20:09.500 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:20:09.529 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:20:09.621 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:20:09.650 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T07:20:09.798 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T07:20:09.856 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T07:20:09.884 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:20:09.922 INFO:teuthology.orchestra.run.smithi159.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 51 kB/s | 42 kB 00:00 2024-09-19T07:20:09.970 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:20:09.993 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T07:20:10.039 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T07:20:10.061 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T07:20:10.094 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T07:20:10.210 INFO:teuthology.orchestra.run.smithi159.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 62 kB/s | 44 kB 00:00 2024-09-19T07:20:10.279 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T07:20:10.424 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T07:20:10.560 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T07:20:10.665 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:20:10.710 INFO:teuthology.orchestra.run.smithi159.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 280 kB/s | 221 kB 00:00 2024-09-19T07:20:10.737 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:20:10.777 INFO:teuthology.orchestra.run.smithi159.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 63 kB/s | 36 kB 00:00 2024-09-19T07:20:10.964 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:20:11.065 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T07:20:11.153 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T07:20:11.192 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T07:20:11.235 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T07:20:11.294 INFO:teuthology.orchestra.run.smithi159.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 81 kB/s | 47 kB 00:00 2024-09-19T07:20:11.299 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T07:20:11.345 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T07:20:11.377 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T07:20:11.409 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-19T07:20:11.429 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T07:20:11.462 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T07:20:11.490 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T07:20:11.534 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T07:20:11.559 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T07:20:11.612 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T07:20:11.643 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T07:20:11.661 INFO:teuthology.orchestra.run.smithi159.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 344 kB/s | 304 kB 00:00 2024-09-19T07:20:11.668 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T07:20:11.687 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T07:20:11.764 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T07:20:11.802 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T07:20:11.846 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T07:20:11.874 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T07:20:11.877 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T07:20:11.919 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T07:20:11.919 INFO:teuthology.orchestra.run.smithi159.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 92 kB/s | 58 kB 00:00 2024-09-19T07:20:11.944 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T07:20:11.945 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T07:20:11.995 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T07:20:11.997 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T07:20:12.072 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T07:20:12.222 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T07:20:12.275 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T07:20:12.339 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T07:20:12.403 INFO:teuthology.orchestra.run.smithi159.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 129 kB/s | 96 kB 00:00 2024-09-19T07:20:12.453 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:20:12.575 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:20:12.581 INFO:teuthology.orchestra.run.smithi159.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 98 kB/s | 64 kB 00:00 2024-09-19T07:20:12.613 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:20:12.670 INFO:teuthology.orchestra.run.smithi159.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 252 kB/s | 22 kB 00:00 2024-09-19T07:20:12.686 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:20:12.725 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T07:20:12.802 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T07:20:12.812 INFO:teuthology.orchestra.run.smithi159.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 6.5 MB/s | 863 kB 00:00 2024-09-19T07:20:12.871 INFO:teuthology.orchestra.run.smithi159.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-19T07:20:12.874 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T07:20:12.886 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T07:20:12.925 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T07:20:12.929 INFO:teuthology.orchestra.run.smithi159.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 5.2 MB/s | 308 kB 00:00 2024-09-19T07:20:12.930 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T07:20:12.955 INFO:teuthology.orchestra.run.smithi159.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 776 kB/s | 19 kB 00:00 2024-09-19T07:20:12.958 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:20:12.967 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T07:20:13.005 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T07:20:13.038 INFO:teuthology.orchestra.run.smithi159.stdout:(107/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-09-19T07:20:13.047 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T07:20:13.062 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:20:13.084 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T07:20:13.088 INFO:teuthology.orchestra.run.smithi159.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 528 kB/s | 26 kB 00:00 2024-09-19T07:20:13.114 INFO:teuthology.orchestra.run.smithi159.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T07:20:13.147 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T07:20:13.147 INFO:teuthology.orchestra.run.smithi159.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-19T07:20:13.169 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T07:20:13.181 INFO:teuthology.orchestra.run.smithi159.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-19T07:20:13.192 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T07:20:13.200 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T07:20:13.259 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T07:20:13.307 INFO:teuthology.orchestra.run.smithi159.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-19T07:20:13.314 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T07:20:13.341 INFO:teuthology.orchestra.run.smithi159.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.2 MB/s | 839 kB 00:00 2024-09-19T07:20:13.374 INFO:teuthology.orchestra.run.smithi159.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-09-19T07:20:13.386 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T07:20:13.386 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T07:20:13.399 INFO:teuthology.orchestra.run.smithi159.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-09-19T07:20:13.425 INFO:teuthology.orchestra.run.smithi159.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-09-19T07:20:13.436 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T07:20:13.450 INFO:teuthology.orchestra.run.smithi159.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-19T07:20:13.484 INFO:teuthology.orchestra.run.smithi159.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-19T07:20:13.488 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T07:20:13.517 INFO:teuthology.orchestra.run.smithi159.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-19T07:20:13.520 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T07:20:13.535 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T07:20:13.571 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T07:20:13.573 INFO:teuthology.orchestra.run.smithi159.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.3 MB/s | 303 kB 00:00 2024-09-19T07:20:13.610 INFO:teuthology.orchestra.run.smithi159.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 2.3 MB/s | 220 kB 00:00 2024-09-19T07:20:13.614 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T07:20:13.642 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T07:20:13.643 INFO:teuthology.orchestra.run.smithi159.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-19T07:20:13.656 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T07:20:13.677 INFO:teuthology.orchestra.run.smithi159.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 325 kB/s | 11 kB 00:00 2024-09-19T07:20:13.702 INFO:teuthology.orchestra.run.smithi159.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-09-19T07:20:13.727 INFO:teuthology.orchestra.run.smithi159.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 940 kB/s | 23 kB 00:00 2024-09-19T07:20:13.738 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:20:13.752 INFO:teuthology.orchestra.run.smithi159.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 784 kB/s | 19 kB 00:00 2024-09-19T07:20:13.778 INFO:teuthology.orchestra.run.smithi159.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 793 kB/s | 20 kB 00:00 2024-09-19T07:20:13.791 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T07:20:13.834 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T07:20:13.836 INFO:teuthology.orchestra.run.smithi159.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.8 MB/s | 2.0 MB 00:00 2024-09-19T07:20:13.838 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:20:13.862 INFO:teuthology.orchestra.run.smithi159.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T07:20:13.875 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T07:20:13.921 INFO:teuthology.orchestra.run.smithi159.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-19T07:20:13.946 INFO:teuthology.orchestra.run.smithi159.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T07:20:13.988 INFO:teuthology.orchestra.run.smithi159.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-09-19T07:20:14.021 INFO:teuthology.orchestra.run.smithi159.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-09-19T07:20:14.038 INFO:teuthology.orchestra.run.smithi159.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-09-19T07:20:14.055 INFO:teuthology.orchestra.run.smithi159.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.7 MB/s | 28 kB 00:00 2024-09-19T07:20:14.059 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:20:14.082 INFO:teuthology.orchestra.run.smithi159.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-19T07:20:14.107 INFO:teuthology.orchestra.run.smithi159.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 595 kB/s | 31 kB 00:00 2024-09-19T07:20:14.133 INFO:teuthology.orchestra.run.smithi159.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T07:20:14.166 INFO:teuthology.orchestra.run.smithi159.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-19T07:20:14.168 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T07:20:14.171 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T07:20:14.192 INFO:teuthology.orchestra.run.smithi159.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 610 kB/s | 36 kB 00:00 2024-09-19T07:20:14.209 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T07:20:14.249 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T07:20:14.250 INFO:teuthology.orchestra.run.smithi159.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 892 kB/s | 74 kB 00:00 2024-09-19T07:20:14.262 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T07:20:14.275 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T07:20:14.275 INFO:teuthology.orchestra.run.smithi159.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-19T07:20:14.299 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T07:20:14.301 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T07:20:14.302 INFO:teuthology.orchestra.run.smithi159.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-19T07:20:14.326 INFO:teuthology.orchestra.run.smithi159.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 808 kB/s | 20 kB 00:00 2024-09-19T07:20:14.336 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T07:20:14.360 INFO:teuthology.orchestra.run.smithi159.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-19T07:20:14.373 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T07:20:14.385 INFO:teuthology.orchestra.run.smithi159.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-19T07:20:14.401 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T07:20:14.444 INFO:teuthology.orchestra.run.smithi159.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-19T07:20:14.445 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T07:20:14.477 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T07:20:14.492 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T07:20:14.524 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T07:20:14.529 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T07:20:14.544 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T07:20:14.561 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T07:20:14.578 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T07:20:14.589 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T07:20:14.617 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T07:20:14.649 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T07:20:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T07:20:14.677 INFO:teuthology.orchestra.run.smithi159.stdout:(148/156): librados2-19.1.1-361.g2f599a46.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-19T07:20:14.679 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T07:20:14.711 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T07:20:14.713 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T07:20:14.742 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T07:20:14.744 INFO:teuthology.orchestra.run.smithi159.stdout:(149/156): librbd1-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-19T07:20:14.746 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T07:20:14.777 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T07:20:14.788 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T07:20:14.823 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T07:20:14.853 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T07:20:14.896 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T07:20:14.909 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T07:20:14.948 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T07:20:14.953 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T07:20:14.977 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T07:20:14.985 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T07:20:15.046 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T07:20:15.087 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T07:20:15.091 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T07:20:15.135 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T07:20:15.166 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T07:20:15.298 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T07:20:15.352 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T07:20:15.845 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T07:20:15.882 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T07:20:15.892 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T07:20:15.935 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T07:20:15.955 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T07:20:15.985 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T07:20:16.032 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T07:20:16.044 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T07:20:16.082 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T07:20:16.089 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T07:20:16.123 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T07:20:16.134 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T07:20:16.213 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T07:20:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T07:20:16.293 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T07:20:16.336 INFO:teuthology.orchestra.run.smithi159.stdout:(150/156): NetworkManager-libnm-1.51.0-1.el9.x8 1.1 MB/s | 1.8 MB 00:01 2024-09-19T07:20:16.368 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T07:20:16.387 INFO:teuthology.orchestra.run.smithi159.stdout:(151/156): NetworkManager-1.51.0-1.el9.x86_64.r 1.4 MB/s | 2.3 MB 00:01 2024-09-19T07:20:16.423 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T07:20:16.431 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T07:20:16.468 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T07:20:16.495 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T07:20:16.527 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T07:20:16.545 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T07:20:16.572 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T07:20:16.618 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T07:20:16.654 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T07:20:16.697 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T07:20:16.737 INFO:teuthology.orchestra.run.smithi159.stdout:(152/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.1 MB/s | 16 MB 00:07 2024-09-19T07:20:16.739 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T07:20:16.874 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T07:20:16.916 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T07:20:16.957 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T07:20:17.068 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T07:20:17.108 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T07:20:17.146 INFO:teuthology.orchestra.run.smithi159.stdout:(153/156): NetworkManager-team-1.51.0-1.el9.x86 51 kB/s | 41 kB 00:00 2024-09-19T07:20:17.147 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T07:20:17.174 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T07:20:17.184 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T07:20:17.198 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T07:20:17.272 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T07:20:17.300 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T07:20:17.346 INFO:teuthology.orchestra.run.smithi159.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 260 kB/s | 249 kB 00:00 2024-09-19T07:20:17.496 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T07:20:17.567 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T07:20:17.600 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T07:20:17.634 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T07:20:17.690 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T07:20:17.704 INFO:teuthology.orchestra.run.smithi159.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:00 2024-09-19T07:20:17.722 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T07:20:17.752 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T07:20:17.786 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T07:20:17.819 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T07:20:17.861 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T07:20:17.912 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T07:20:17.999 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T07:20:18.051 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T07:20:18.090 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T07:20:18.165 INFO:teuthology.orchestra.run.smithi159.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-09-19T07:20:18.171 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-19T07:20:18.171 INFO:teuthology.orchestra.run.smithi159.stdout:Total 7.4 MB/s | 226 MB 00:30 2024-09-19T07:20:18.201 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T07:20:18.258 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T07:20:18.510 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T07:20:18.591 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T07:20:18.619 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T07:20:18.652 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T07:20:18.961 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T07:20:18.980 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T07:20:19.007 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T07:20:19.030 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T07:20:19.064 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T07:20:19.082 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T07:20:19.112 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T07:20:19.148 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T07:20:19.162 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T07:20:19.220 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T07:20:19.274 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T07:20:19.369 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T07:20:19.410 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T07:20:19.595 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T07:20:19.627 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T07:20:19.632 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T07:20:19.669 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T07:20:19.709 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T07:20:19.723 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T07:20:19.746 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T07:20:20.233 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-19T07:20:20.344 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T07:20:20.413 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-19T07:20:20.413 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-19T07:20:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T07:20:21.665 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T07:20:21.744 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T07:20:21.771 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T07:20:21.805 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T07:20:22.065 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:20:22.101 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:20:22.117 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T07:20:22.167 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T07:20:22.258 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T07:20:22.298 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T07:20:22.333 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T07:20:22.645 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:20:22.680 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T07:20:22.758 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-19T07:20:22.759 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-19T07:20:22.789 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T07:20:22.789 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T07:20:22.789 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T07:20:22.789 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:20:22.819 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T07:20:22.823 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T07:20:22.854 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T07:20:22.925 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T07:20:22.955 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T07:20:25.256 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:20:25.320 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:20:25.868 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:20:25.907 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T07:20:25.995 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T07:20:25.995 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T07:20:25.995 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T07:20:25.995 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:20:26.033 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T07:20:29.604 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-19T07:20:29.743 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T07:20:29.827 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T07:20:29.898 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T07:20:29.938 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T07:20:29.969 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T07:20:30.029 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T07:20:30.398 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T07:20:30.528 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:20:30.664 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:20:30.695 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:20:30.798 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:20:30.841 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T07:20:31.062 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T07:20:31.128 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T07:20:31.183 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:20:31.310 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:20:31.350 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T07:20:31.432 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T07:20:31.477 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T07:20:31.522 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T07:20:31.719 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T07:20:31.922 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T07:20:32.158 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T07:20:32.286 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:20:32.393 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:20:32.698 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:20:32.835 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T07:20:32.950 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T07:20:33.033 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T07:20:33.077 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T07:20:33.159 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T07:20:33.221 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T07:20:33.260 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T07:20:33.316 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T07:20:33.353 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T07:20:33.387 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T07:20:33.475 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T07:20:33.547 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T07:20:33.583 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T07:20:33.629 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T07:20:33.710 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T07:20:33.753 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T07:20:33.820 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T07:20:33.858 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T07:20:33.949 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T07:20:34.002 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T07:20:34.103 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T07:20:34.293 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T07:20:34.355 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T07:20:34.696 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T07:20:34.696 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /sys 2024-09-19T07:20:34.696 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /proc 2024-09-19T07:20:34.696 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /mnt 2024-09-19T07:20:34.696 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /var/tmp 2024-09-19T07:20:34.696 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /home 2024-09-19T07:20:34.696 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /root 2024-09-19T07:20:34.696 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /tmp 2024-09-19T07:20:34.696 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:20:34.873 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T07:20:34.888 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T07:20:34.889 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:34.889 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T07:20:34.889 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:20:34.990 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T07:20:35.096 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T07:20:35.150 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T07:20:35.194 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T07:20:35.232 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T07:20:35.285 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T07:20:35.458 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T07:20:35.527 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T07:20:35.594 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T07:20:35.680 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T07:20:35.719 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T07:20:35.756 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T07:20:35.840 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T07:20:35.880 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T07:20:35.925 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T07:20:35.975 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T07:20:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T07:20:36.200 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T07:20:36.243 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T07:20:36.367 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T07:20:36.376 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T07:20:36.402 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T07:20:36.402 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:36.402 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T07:20:36.402 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T07:20:36.402 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T07:20:36.403 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:20:36.415 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T07:20:36.480 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T07:20:36.509 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T07:20:36.534 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T07:20:36.635 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T07:20:36.650 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T07:20:36.658 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T07:20:36.658 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:36.659 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T07:20:36.659 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T07:20:36.659 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T07:20:36.659 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:20:36.810 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T07:20:36.850 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T07:20:36.888 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T07:20:36.929 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T07:20:36.978 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T07:20:37.026 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T07:20:37.063 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T07:20:37.094 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T07:20:37.118 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T07:20:37.118 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:37.118 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T07:20:37.118 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T07:20:37.118 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T07:20:37.118 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:20:37.123 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T07:20:37.156 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T07:20:37.200 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T07:20:37.232 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T07:20:37.233 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T07:20:37.290 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T07:20:37.331 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T07:20:37.345 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T07:20:37.382 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T07:20:37.452 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T07:20:37.538 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T07:20:37.583 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T07:20:37.587 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T07:20:37.648 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T07:20:37.665 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T07:20:37.753 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T07:20:37.765 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T07:20:37.765 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /sys 2024-09-19T07:20:37.766 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /proc 2024-09-19T07:20:37.766 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /mnt 2024-09-19T07:20:37.766 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /var/tmp 2024-09-19T07:20:37.766 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /home 2024-09-19T07:20:37.766 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /root 2024-09-19T07:20:37.766 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /tmp 2024-09-19T07:20:37.766 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:20:37.894 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T07:20:37.934 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T07:20:37.946 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T07:20:37.953 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T07:20:37.953 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:37.953 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T07:20:37.953 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:20:37.978 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T07:20:38.031 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T07:20:38.476 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T07:20:38.505 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T07:20:38.534 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T07:20:38.537 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T07:20:38.594 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T07:20:38.599 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T07:20:38.628 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T07:20:38.637 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T07:20:38.741 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T07:20:38.811 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T07:20:38.872 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T07:20:38.969 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T07:20:39.013 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T07:20:39.251 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T07:20:39.285 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T07:20:39.383 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T07:20:39.429 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T07:20:39.456 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T07:20:39.456 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:39.456 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T07:20:39.456 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T07:20:39.456 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T07:20:39.456 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:20:39.679 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T07:20:39.688 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T07:20:39.711 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T07:20:39.711 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:39.711 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T07:20:39.711 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T07:20:39.711 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T07:20:39.711 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:20:39.714 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T07:20:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T07:20:40.170 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T07:20:40.170 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:40.170 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T07:20:40.170 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T07:20:40.170 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T07:20:40.170 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:20:40.231 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T07:20:40.268 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T07:20:40.317 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T07:20:40.381 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T07:20:40.389 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T07:20:40.573 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T07:20:40.622 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T07:20:40.671 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T07:20:40.692 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T07:20:40.698 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T07:20:40.788 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T07:20:40.798 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T07:20:40.941 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T07:20:40.960 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T07:20:40.981 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T07:20:40.992 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T07:20:41.013 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T07:20:41.015 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T07:20:41.015 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:41.015 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T07:20:41.015 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T07:20:41.015 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T07:20:41.016 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:20:41.048 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T07:20:41.059 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T07:20:41.071 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T07:20:41.168 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T07:20:41.216 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T07:20:41.460 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T07:20:41.497 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T07:20:41.529 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T07:20:41.591 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T07:20:41.620 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T07:20:41.970 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T07:20:42.098 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T07:20:42.135 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T07:20:42.139 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T07:20:42.168 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T07:20:42.179 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T07:20:42.202 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T07:20:42.202 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:42.202 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T07:20:42.202 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T07:20:42.202 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T07:20:42.202 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:20:42.246 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T07:20:42.269 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T07:20:42.269 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:42.269 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T07:20:42.269 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:20:42.314 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T07:20:42.337 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T07:20:42.337 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:42.337 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T07:20:42.337 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T07:20:42.337 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T07:20:42.337 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:20:42.504 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T07:20:42.554 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T07:20:42.639 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T07:20:42.663 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T07:20:42.663 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:42.663 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T07:20:42.663 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T07:20:42.663 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T07:20:42.663 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:20:42.665 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T07:20:42.670 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T07:20:42.705 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T07:20:42.733 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T07:20:42.784 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T07:20:43.315 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T07:20:43.376 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T07:20:43.473 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T07:20:43.500 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T07:20:43.668 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T07:20:43.697 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T07:20:43.798 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T07:20:43.967 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T07:20:43.999 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T07:20:44.023 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T07:20:44.023 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:44.024 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T07:20:44.024 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T07:20:44.024 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T07:20:44.024 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:20:44.075 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T07:20:44.087 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T07:20:44.184 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T07:20:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T07:20:45.157 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T07:20:45.194 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T07:20:45.218 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T07:20:45.218 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:45.218 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T07:20:45.218 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T07:20:45.218 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T07:20:45.218 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:20:45.270 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T07:20:45.293 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T07:20:45.293 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:45.293 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T07:20:45.293 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:20:45.345 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T07:20:45.369 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T07:20:45.369 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:45.369 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T07:20:45.369 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T07:20:45.369 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T07:20:45.369 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:20:45.671 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T07:20:45.695 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T07:20:45.695 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:20:45.695 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T07:20:45.695 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T07:20:45.695 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T07:20:45.695 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:20:45.985 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:20:46.056 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:20:46.681 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:20:46.733 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T07:20:46.843 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T07:20:46.843 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T07:20:46.844 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T07:20:46.844 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:20:46.886 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T07:20:47.211 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T07:20:47.252 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T07:20:47.293 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T07:20:47.326 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T07:20:47.353 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T07:20:47.533 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T07:20:47.575 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T07:20:47.605 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T07:20:47.635 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T07:20:47.692 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T07:20:48.328 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:20:48.372 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:20:48.392 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:20:49.231 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T07:20:49.319 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T07:20:49.361 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T07:20:49.442 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T07:20:49.473 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T07:20:49.498 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T07:20:49.525 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T07:20:49.525 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T07:20:49.609 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T07:20:49.609 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T07:20:49.685 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T07:20:49.718 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T07:20:49.723 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:20:49.723 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:20:49.903 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:20:49.903 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T07:20:49.987 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T07:20:50.051 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:20:50.249 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T07:20:50.307 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T07:20:50.348 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T07:20:50.380 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T07:20:50.407 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T07:20:50.595 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T07:20:50.646 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T07:20:50.676 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T07:20:50.706 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T07:20:50.752 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T07:20:51.366 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:20:51.407 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:20:51.428 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:20:52.283 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T07:20:52.371 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T07:20:52.412 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T07:20:52.501 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T07:20:52.531 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T07:20:52.557 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T07:20:52.583 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T07:20:52.583 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T07:20:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T07:20:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T07:20:52.743 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T07:20:52.776 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T07:20:52.781 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:20:52.781 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:20:52.970 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:20:52.970 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T07:20:53.054 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T07:20:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:20:56.088 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:20:56.088 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T07:20:56.088 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T07:20:56.089 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T07:20:56.089 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T07:20:56.089 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T07:20:56.089 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T07:20:56.089 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T07:20:56.089 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:20:56.089 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:20:56.089 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T07:20:56.089 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T07:20:56.089 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T07:20:56.089 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:20:56.089 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T07:20:56.089 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T07:20:56.090 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T07:20:56.090 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T07:20:56.090 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T07:20:56.090 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T07:20:56.090 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T07:20:56.090 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T07:20:56.090 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T07:20:56.090 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T07:20:56.090 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T07:20:56.090 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T07:20:56.090 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T07:20:56.090 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T07:20:56.091 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T07:20:56.091 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T07:20:56.091 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T07:20:56.091 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T07:20:56.091 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T07:20:56.091 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T07:20:56.091 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T07:20:56.091 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T07:20:56.091 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T07:20:56.091 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T07:20:56.091 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T07:20:56.091 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T07:20:56.092 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T07:20:56.092 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T07:20:56.092 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T07:20:56.092 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T07:20:56.092 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T07:20:56.092 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T07:20:56.092 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T07:20:56.092 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T07:20:56.092 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T07:20:56.092 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T07:20:56.092 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T07:20:56.092 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T07:20:56.092 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T07:20:56.093 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T07:20:56.093 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T07:20:56.093 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T07:20:56.093 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T07:20:56.093 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T07:20:56.093 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T07:20:56.093 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T07:20:56.093 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T07:20:56.093 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T07:20:56.093 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T07:20:56.093 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T07:20:56.093 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T07:20:56.093 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T07:20:56.094 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T07:20:56.094 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T07:20:56.094 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T07:20:56.094 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T07:20:56.094 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T07:20:56.094 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T07:20:56.094 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T07:20:56.094 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T07:20:56.094 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T07:20:56.094 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T07:20:56.094 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T07:20:56.094 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T07:20:56.094 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T07:20:56.095 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T07:20:56.095 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T07:20:56.095 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T07:20:56.095 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T07:20:56.095 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T07:20:56.095 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T07:20:56.095 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T07:20:56.095 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T07:20:56.095 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T07:20:56.095 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T07:20:56.095 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T07:20:56.095 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T07:20:56.095 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T07:20:56.096 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T07:20:56.096 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T07:20:56.096 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T07:20:56.096 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T07:20:56.096 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T07:20:56.096 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T07:20:56.096 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T07:20:56.096 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T07:20:56.096 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T07:20:56.096 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T07:20:56.096 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T07:20:56.096 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T07:20:56.097 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T07:20:56.097 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T07:20:56.097 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T07:20:56.097 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T07:20:56.097 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T07:20:56.097 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T07:20:56.097 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T07:20:56.097 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T07:20:56.097 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T07:20:56.097 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T07:20:56.097 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T07:20:56.097 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T07:20:56.098 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T07:20:56.098 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T07:20:56.098 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T07:20:56.098 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T07:20:56.098 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T07:20:56.098 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T07:20:56.098 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T07:20:56.098 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T07:20:56.098 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T07:20:56.098 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T07:20:56.098 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T07:20:56.098 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T07:20:56.098 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T07:20:56.099 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T07:20:56.099 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T07:20:56.099 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T07:20:56.099 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T07:20:56.099 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T07:20:56.099 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T07:20:56.099 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T07:20:56.099 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T07:20:56.099 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T07:20:56.099 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T07:20:56.099 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T07:20:56.099 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T07:20:56.099 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T07:20:56.100 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T07:20:56.100 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T07:20:56.100 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T07:20:56.100 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T07:20:56.100 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T07:20:56.100 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T07:20:56.100 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T07:20:56.100 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T07:20:56.100 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T07:20:56.100 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T07:20:56.100 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T07:20:56.100 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T07:20:56.100 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T07:20:56.100 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T07:20:56.101 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T07:20:56.101 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T07:20:56.101 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T07:20:56.101 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T07:20:56.101 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T07:20:56.101 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T07:20:56.101 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T07:20:56.101 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T07:20:58.638 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:20:58.638 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T07:20:58.638 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T07:20:58.638 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T07:20:58.639 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T07:20:58.639 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T07:20:58.639 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T07:20:58.639 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T07:20:58.639 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:20:58.639 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:20:58.639 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T07:20:58.639 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T07:20:58.639 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T07:20:58.639 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:20:58.639 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T07:20:58.639 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T07:20:58.639 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T07:20:58.639 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T07:20:58.640 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T07:20:58.640 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T07:20:58.640 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T07:20:58.640 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T07:20:58.640 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T07:20:58.640 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T07:20:58.640 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T07:20:58.640 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T07:20:58.640 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T07:20:58.640 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T07:20:58.652 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T07:20:58.653 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T07:20:58.653 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T07:20:58.653 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T07:20:58.653 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T07:20:58.653 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T07:20:58.653 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T07:20:58.653 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T07:20:58.653 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T07:20:58.653 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T07:20:58.653 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T07:20:58.653 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T07:20:58.653 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T07:20:58.653 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T07:20:58.654 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T07:20:58.654 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T07:20:58.654 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T07:20:58.654 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T07:20:58.654 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T07:20:58.654 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T07:20:58.654 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T07:20:58.654 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T07:20:58.654 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T07:20:58.654 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T07:20:58.654 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T07:20:58.654 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T07:20:58.654 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T07:20:58.654 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T07:20:58.654 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T07:20:58.655 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T07:20:58.655 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T07:20:58.655 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T07:20:58.655 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T07:20:58.655 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T07:20:58.655 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T07:20:58.655 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T07:20:58.655 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T07:20:58.655 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T07:20:58.655 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T07:20:58.655 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T07:20:58.655 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T07:20:58.655 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T07:20:58.656 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T07:20:58.656 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T07:20:58.656 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T07:20:58.656 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T07:20:58.656 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T07:20:58.656 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T07:20:58.656 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T07:20:58.656 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T07:20:58.656 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T07:20:58.656 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T07:20:58.656 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T07:20:58.656 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T07:20:58.656 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T07:20:58.656 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T07:20:58.657 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T07:20:58.657 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T07:20:58.657 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T07:20:58.657 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T07:20:58.657 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T07:20:58.657 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T07:20:58.657 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T07:20:58.657 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T07:20:58.657 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T07:20:58.657 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T07:20:58.657 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T07:20:58.657 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T07:20:58.657 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T07:20:58.657 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T07:20:58.657 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T07:20:58.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T07:20:58.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T07:20:58.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T07:20:58.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T07:20:58.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T07:20:58.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T07:20:58.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T07:20:58.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T07:20:58.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T07:20:58.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T07:20:58.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T07:20:58.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T07:20:58.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T07:20:58.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T07:20:58.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T07:20:58.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T07:20:58.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T07:20:58.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T07:20:58.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T07:20:58.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T07:20:58.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T07:20:58.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T07:20:58.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T07:20:58.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T07:20:58.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T07:20:58.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T07:20:58.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T07:20:58.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T07:20:58.660 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T07:20:58.660 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T07:20:58.660 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T07:20:58.660 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T07:20:58.660 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T07:20:58.660 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T07:20:58.660 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T07:20:58.660 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T07:20:58.660 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T07:20:58.660 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T07:20:58.660 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T07:20:58.660 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T07:20:58.660 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T07:20:58.660 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T07:20:58.661 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T07:20:58.661 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T07:20:58.661 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T07:20:58.661 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T07:20:58.661 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T07:20:58.661 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T07:20:58.661 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T07:20:58.661 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T07:20:58.661 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T07:20:58.661 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T07:20:58.661 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T07:20:58.661 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T07:20:58.661 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T07:20:58.662 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T07:20:58.662 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T07:20:58.662 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T07:20:58.662 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T07:20:58.662 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T07:20:58.662 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T07:20:58.662 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T07:20:58.662 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T07:20:58.662 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T07:20:58.662 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T07:20:58.696 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:20:58.696 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:20:58.696 INFO:teuthology.orchestra.run.smithi203.stdout:Upgraded: 2024-09-19T07:20:58.696 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T07:20:58.696 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T07:20:58.696 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T07:20:58.696 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T07:20:58.696 INFO:teuthology.orchestra.run.smithi203.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.697 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.697 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T07:20:58.697 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T07:20:58.697 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-09-19T07:20:58.697 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T07:20:58.697 INFO:teuthology.orchestra.run.smithi203.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T07:20:58.697 INFO:teuthology.orchestra.run.smithi203.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T07:20:58.697 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T07:20:58.697 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.697 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.697 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.697 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.697 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:20:58.698 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.698 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.698 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.698 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:20:58.698 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:20:58.698 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:20:58.698 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:20:58.698 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:20:58.698 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.698 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.698 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:20:58.698 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.698 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.698 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.699 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:20:58.699 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:20:58.699 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.699 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.699 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:20:58.699 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T07:20:58.699 INFO:teuthology.orchestra.run.smithi203.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T07:20:58.699 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T07:20:58.699 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T07:20:58.699 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T07:20:58.699 INFO:teuthology.orchestra.run.smithi203.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T07:20:58.699 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T07:20:58.699 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T07:20:58.699 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T07:20:58.700 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T07:20:58.700 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T07:20:58.700 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.700 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.700 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.700 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T07:20:58.700 INFO:teuthology.orchestra.run.smithi203.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T07:20:58.700 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T07:20:58.700 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T07:20:58.700 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T07:20:58.700 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.700 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.700 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T07:20:58.700 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.701 INFO:teuthology.orchestra.run.smithi203.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T07:20:58.701 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T07:20:58.701 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T07:20:58.701 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T07:20:58.701 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T07:20:58.701 INFO:teuthology.orchestra.run.smithi203.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T07:20:58.701 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T07:20:58.701 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T07:20:58.701 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T07:20:58.701 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T07:20:58.701 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T07:20:58.701 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T07:20:58.701 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T07:20:58.701 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T07:20:58.702 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T07:20:58.702 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T07:20:58.702 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T07:20:58.702 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T07:20:58.702 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T07:20:58.702 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T07:20:58.702 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T07:20:58.702 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T07:20:58.702 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T07:20:58.702 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.702 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.702 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.702 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T07:20:58.702 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T07:20:58.703 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T07:20:58.703 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T07:20:58.703 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T07:20:58.703 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T07:20:58.703 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T07:20:58.703 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T07:20:58.703 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T07:20:58.703 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T07:20:58.703 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T07:20:58.703 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T07:20:58.703 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T07:20:58.703 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T07:20:58.703 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T07:20:58.703 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T07:20:58.703 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T07:20:58.704 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T07:20:58.704 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T07:20:58.704 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T07:20:58.704 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T07:20:58.704 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T07:20:58.704 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T07:20:58.704 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T07:20:58.704 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T07:20:58.704 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T07:20:58.704 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T07:20:58.704 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T07:20:58.704 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T07:20:58.704 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T07:20:58.704 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T07:20:58.705 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T07:20:58.705 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T07:20:58.705 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T07:20:58.705 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T07:20:58.705 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T07:20:58.705 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T07:20:58.705 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T07:20:58.705 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T07:20:58.705 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T07:20:58.705 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.705 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.705 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T07:20:58.705 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T07:20:58.705 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T07:20:58.706 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.706 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T07:20:58.706 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T07:20:58.706 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T07:20:58.706 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T07:20:58.706 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T07:20:58.706 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T07:20:58.706 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T07:20:58.706 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T07:20:58.706 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T07:20:58.706 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T07:20:58.706 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T07:20:58.706 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T07:20:58.706 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T07:20:58.706 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T07:20:58.707 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T07:20:58.707 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.707 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.707 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:20:58.707 INFO:teuthology.orchestra.run.smithi203.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T07:20:58.707 INFO:teuthology.orchestra.run.smithi203.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T07:20:58.707 INFO:teuthology.orchestra.run.smithi203.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T07:20:58.707 INFO:teuthology.orchestra.run.smithi203.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T07:20:58.707 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T07:20:58.707 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T07:20:58.708 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T07:20:58.708 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:20:58.708 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-09-19T07:20:59.153 DEBUG:teuthology.parallel:result is None 2024-09-19T07:20:59.773 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T07:20:59.773 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /sys 2024-09-19T07:20:59.773 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /proc 2024-09-19T07:20:59.773 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /mnt 2024-09-19T07:20:59.773 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /var/tmp 2024-09-19T07:20:59.773 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /home 2024-09-19T07:20:59.774 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /root 2024-09-19T07:20:59.774 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /tmp 2024-09-19T07:20:59.774 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:20:59.980 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T07:21:00.018 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T07:21:00.018 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:21:00.018 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T07:21:00.018 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:21:01.164 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:21:01.164 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:21:01.164 INFO:teuthology.orchestra.run.smithi189.stdout:Upgraded: 2024-09-19T07:21:01.164 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T07:21:01.164 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T07:21:01.164 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T07:21:01.164 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T07:21:01.164 INFO:teuthology.orchestra.run.smithi189.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.164 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.164 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T07:21:01.164 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T07:21:01.165 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-19T07:21:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T07:21:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T07:21:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T07:21:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T07:21:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T07:21:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T07:21:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T07:21:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T07:21:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T07:21:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T07:21:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T07:21:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T07:21:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T07:21:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T07:21:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T07:21:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T07:21:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T07:21:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T07:21:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T07:21:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T07:21:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T07:21:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T07:21:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T07:21:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T07:21:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T07:21:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T07:21:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T07:21:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T07:21:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T07:21:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T07:21:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T07:21:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T07:21:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T07:21:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T07:21:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T07:21:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T07:21:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T07:21:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T07:21:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T07:21:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T07:21:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T07:21:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T07:21:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T07:21:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T07:21:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T07:21:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T07:21:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T07:21:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T07:21:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T07:21:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T07:21:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T07:21:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T07:21:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T07:21:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T07:21:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T07:21:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T07:21:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T07:21:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T07:21:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T07:21:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T07:21:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T07:21:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T07:21:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T07:21:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T07:21:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T07:21:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T07:21:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T07:21:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T07:21:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T07:21:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T07:21:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T07:21:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T07:21:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T07:21:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T07:21:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T07:21:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T07:21:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T07:21:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T07:21:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T07:21:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T07:21:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T07:21:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T07:21:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T07:21:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T07:21:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T07:21:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T07:21:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T07:21:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T07:21:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T07:21:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T07:21:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T07:21:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T07:21:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T07:21:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T07:21:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T07:21:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T07:21:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T07:21:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T07:21:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T07:21:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T07:21:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T07:21:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T07:21:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T07:21:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T07:21:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T07:21:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T07:21:01.175 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T07:21:01.175 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T07:21:01.175 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T07:21:01.175 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:21:01.175 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-19T07:21:01.606 DEBUG:teuthology.parallel:result is None 2024-09-19T07:21:01.635 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T07:21:01.693 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T07:21:01.693 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:21:01.693 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T07:21:01.693 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T07:21:01.693 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T07:21:01.693 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:21:01.944 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T07:21:01.994 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T07:21:01.994 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:21:01.994 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T07:21:01.994 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T07:21:01.994 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T07:21:01.995 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:21:02.469 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T07:21:02.519 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T07:21:02.520 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:21:02.520 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T07:21:02.520 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T07:21:02.520 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T07:21:02.520 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:21:02.652 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T07:21:02.729 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T07:21:02.793 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T07:21:03.019 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T07:21:03.081 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T07:21:03.166 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T07:21:03.277 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T07:21:03.494 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T07:21:03.551 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T07:21:03.596 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T07:21:03.621 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T07:21:04.187 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T07:21:04.220 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T07:21:04.257 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T07:21:04.321 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T07:21:04.376 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T07:21:05.608 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T07:21:05.664 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T07:21:06.759 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T07:21:06.795 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T07:21:06.933 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T07:21:07.131 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T07:21:07.163 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T07:21:07.215 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T07:21:07.216 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:21:07.216 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T07:21:07.216 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T07:21:07.216 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T07:21:07.216 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:21:07.273 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T07:21:07.299 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T07:21:07.430 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T07:21:07.501 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T07:21:08.505 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T07:21:08.541 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T07:21:08.593 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T07:21:08.593 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:21:08.593 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T07:21:08.593 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T07:21:08.593 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T07:21:08.593 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:21:08.659 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T07:21:08.709 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T07:21:08.709 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:21:08.709 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T07:21:08.709 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:21:08.760 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T07:21:08.811 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T07:21:08.811 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:21:08.811 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T07:21:08.811 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T07:21:08.811 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T07:21:08.811 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:21:09.143 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T07:21:09.195 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T07:21:09.195 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:21:09.195 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T07:21:09.195 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T07:21:09.195 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T07:21:09.195 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:21:14.140 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T07:21:14.181 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T07:21:14.252 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T07:21:14.287 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T07:21:14.323 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T07:21:14.574 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T07:21:14.628 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T07:21:14.657 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T07:21:14.697 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T07:21:14.741 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T07:21:15.557 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:21:15.605 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:21:15.648 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:21:16.945 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T07:21:17.046 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T07:21:17.102 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T07:21:17.210 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T07:21:17.248 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T07:21:17.282 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T07:21:17.319 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T07:21:17.319 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T07:21:17.426 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T07:21:17.426 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T07:21:17.568 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T07:21:17.592 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T07:21:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:21:17.609 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:21:17.874 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:21:17.874 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T07:21:17.971 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T07:21:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:21:18.257 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:21:18.257 INFO:teuthology.orchestra.run.smithi159.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-19T07:21:18.257 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:21:24.349 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T07:21:24.350 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T07:21:24.350 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T07:21:24.350 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T07:21:24.350 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T07:21:24.350 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T07:21:24.350 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T07:21:24.350 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:21:24.350 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:21:24.350 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T07:21:24.350 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T07:21:24.350 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T07:21:24.350 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:21:24.351 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T07:21:24.351 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T07:21:24.351 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T07:21:24.351 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T07:21:24.351 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T07:21:24.351 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T07:21:24.351 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T07:21:24.351 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T07:21:24.351 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T07:21:24.351 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T07:21:24.351 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T07:21:24.351 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T07:21:24.351 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T07:21:24.351 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T07:21:24.352 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T07:21:24.352 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T07:21:24.352 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T07:21:24.352 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T07:21:24.352 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T07:21:24.352 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T07:21:24.352 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T07:21:24.352 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T07:21:24.352 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T07:21:24.352 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T07:21:24.352 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T07:21:24.352 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T07:21:24.353 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T07:21:24.353 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T07:21:24.353 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T07:21:24.353 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T07:21:24.353 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T07:21:24.353 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T07:21:24.353 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T07:21:24.353 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T07:21:24.353 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T07:21:24.353 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T07:21:24.353 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T07:21:24.353 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T07:21:24.353 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T07:21:24.353 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T07:21:24.354 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T07:21:24.354 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T07:21:24.354 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T07:21:24.354 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T07:21:24.354 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T07:21:24.354 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T07:21:24.354 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T07:21:24.354 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T07:21:24.354 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T07:21:24.354 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T07:21:24.354 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T07:21:24.354 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T07:21:24.354 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T07:21:24.354 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T07:21:24.355 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T07:21:24.355 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T07:21:24.355 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T07:21:24.355 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T07:21:24.355 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T07:21:24.355 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T07:21:24.355 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T07:21:24.355 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T07:21:24.355 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T07:21:24.355 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T07:21:24.355 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T07:21:24.355 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T07:21:24.355 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T07:21:24.356 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T07:21:24.356 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T07:21:24.356 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T07:21:24.356 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T07:21:24.356 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T07:21:24.356 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T07:21:24.356 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T07:21:24.356 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T07:21:24.356 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T07:21:24.356 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T07:21:24.356 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T07:21:24.356 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T07:21:24.356 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T07:21:24.356 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T07:21:24.357 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T07:21:24.357 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T07:21:24.357 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T07:21:24.357 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T07:21:24.357 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T07:21:24.357 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T07:21:24.357 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T07:21:24.357 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T07:21:24.357 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T07:21:24.357 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T07:21:24.357 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T07:21:24.357 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T07:21:24.357 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T07:21:24.358 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T07:21:24.358 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T07:21:24.358 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T07:21:24.358 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T07:21:24.358 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T07:21:24.358 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T07:21:24.358 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T07:21:24.358 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T07:21:24.358 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T07:21:24.358 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T07:21:24.358 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T07:21:24.358 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T07:21:24.358 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T07:21:24.358 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T07:21:24.359 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T07:21:24.359 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T07:21:24.359 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T07:21:24.359 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T07:21:24.359 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T07:21:24.359 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T07:21:24.359 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T07:21:24.359 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T07:21:24.359 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T07:21:24.359 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T07:21:24.359 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T07:21:24.359 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T07:21:24.359 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T07:21:24.359 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T07:21:24.359 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T07:21:24.360 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T07:21:24.360 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T07:21:24.360 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T07:21:24.360 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T07:21:24.360 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T07:21:24.360 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T07:21:24.360 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T07:21:24.360 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T07:21:24.360 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T07:21:24.360 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T07:21:24.360 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T07:21:24.360 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T07:21:24.360 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T07:21:24.361 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T07:21:24.361 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T07:21:24.361 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T07:21:24.361 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T07:21:24.361 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T07:21:24.361 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T07:21:24.361 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T07:21:24.361 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T07:21:24.361 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T07:21:24.361 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T07:21:24.361 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T07:21:24.361 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T07:21:24.361 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T07:21:24.361 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T07:21:26.657 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:21:26.657 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:21:26.657 INFO:teuthology.orchestra.run.smithi159.stdout:Upgraded: 2024-09-19T07:21:26.657 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T07:21:26.657 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T07:21:26.657 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T07:21:26.657 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T07:21:26.657 INFO:teuthology.orchestra.run.smithi159.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.657 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.658 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T07:21:26.658 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T07:21:26.658 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-19T07:21:26.658 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T07:21:26.658 INFO:teuthology.orchestra.run.smithi159.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T07:21:26.658 INFO:teuthology.orchestra.run.smithi159.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T07:21:26.658 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T07:21:26.658 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.658 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.658 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.658 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.658 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:26.658 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.659 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.659 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.659 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:26.659 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:26.659 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:26.659 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:26.659 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:26.659 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.659 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.659 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:26.659 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.659 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.659 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.659 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:26.660 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:26.660 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.660 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.660 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:21:26.660 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T07:21:26.660 INFO:teuthology.orchestra.run.smithi159.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T07:21:26.660 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T07:21:26.660 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T07:21:26.660 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T07:21:26.660 INFO:teuthology.orchestra.run.smithi159.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T07:21:26.660 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T07:21:26.660 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T07:21:26.660 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T07:21:26.660 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T07:21:26.661 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T07:21:26.661 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.661 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.661 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.661 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T07:21:26.661 INFO:teuthology.orchestra.run.smithi159.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T07:21:26.661 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T07:21:26.661 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T07:21:26.661 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T07:21:26.661 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.661 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.661 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T07:21:26.661 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.662 INFO:teuthology.orchestra.run.smithi159.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T07:21:26.662 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T07:21:26.662 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T07:21:26.662 INFO:teuthology.orchestra.run.smithi159.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T07:21:26.662 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T07:21:26.662 INFO:teuthology.orchestra.run.smithi159.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T07:21:26.662 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T07:21:26.662 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T07:21:26.662 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T07:21:26.662 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T07:21:26.662 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T07:21:26.662 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T07:21:26.662 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T07:21:26.662 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T07:21:26.662 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T07:21:26.663 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T07:21:26.663 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T07:21:26.663 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T07:21:26.663 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T07:21:26.663 INFO:teuthology.orchestra.run.smithi159.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T07:21:26.663 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T07:21:26.663 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T07:21:26.663 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T07:21:26.663 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.663 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.663 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.663 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T07:21:26.663 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T07:21:26.663 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T07:21:26.664 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T07:21:26.664 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T07:21:26.664 INFO:teuthology.orchestra.run.smithi159.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T07:21:26.664 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T07:21:26.664 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T07:21:26.664 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T07:21:26.664 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T07:21:26.664 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T07:21:26.664 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T07:21:26.664 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T07:21:26.664 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T07:21:26.664 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T07:21:26.664 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T07:21:26.665 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T07:21:26.665 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T07:21:26.665 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T07:21:26.665 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T07:21:26.665 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T07:21:26.665 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T07:21:26.665 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T07:21:26.665 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T07:21:26.665 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T07:21:26.665 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T07:21:26.665 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T07:21:26.665 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T07:21:26.665 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T07:21:26.665 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T07:21:26.665 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T07:21:26.666 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T07:21:26.666 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T07:21:26.666 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T07:21:26.666 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T07:21:26.666 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T07:21:26.666 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T07:21:26.666 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T07:21:26.666 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T07:21:26.666 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T07:21:26.666 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.666 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.666 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T07:21:26.666 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T07:21:26.666 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T07:21:26.667 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.667 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T07:21:26.667 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T07:21:26.667 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T07:21:26.667 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T07:21:26.667 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T07:21:26.667 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T07:21:26.667 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T07:21:26.667 INFO:teuthology.orchestra.run.smithi159.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T07:21:26.667 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T07:21:26.667 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T07:21:26.667 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T07:21:26.667 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T07:21:26.667 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T07:21:26.667 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T07:21:26.668 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T07:21:26.668 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.668 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.668 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:21:26.668 INFO:teuthology.orchestra.run.smithi159.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T07:21:26.668 INFO:teuthology.orchestra.run.smithi159.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T07:21:26.668 INFO:teuthology.orchestra.run.smithi159.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T07:21:26.668 INFO:teuthology.orchestra.run.smithi159.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T07:21:26.668 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T07:21:26.668 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T07:21:26.669 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T07:21:26.669 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:21:26.669 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-19T07:21:27.368 DEBUG:teuthology.parallel:result is None 2024-09-19T07:21:27.369 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:21:27.511 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T07:21:27.562 INFO:teuthology.orchestra.run.smithi159.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T07:21:27.562 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T07:21:27.562 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T07:21:27.565 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:21:27.700 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T07:21:27.727 INFO:teuthology.orchestra.run.smithi189.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T07:21:27.727 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T07:21:27.727 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T07:21:27.729 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:21:27.874 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T07:21:27.901 INFO:teuthology.orchestra.run.smithi203.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T07:21:27.901 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T07:21:27.901 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T07:21:27.903 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T07:21:27.903 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:21:27.903 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T07:21:27.964 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-19T07:21:27.964 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T07:21:27.998 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-19T07:21:27.998 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T07:21:28.031 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T07:21:28.031 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:21:28.031 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T07:21:28.088 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T07:21:28.195 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-19T07:21:28.195 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T07:21:28.224 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T07:21:28.301 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-19T07:21:28.301 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T07:21:28.334 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T07:21:28.405 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T07:21:28.405 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:21:28.405 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T07:21:28.468 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T07:21:28.570 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-19T07:21:28.570 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T07:21:28.596 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T07:21:28.664 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-19T07:21:28.664 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T07:21:28.690 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T07:21:28.757 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T07:21:28.757 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:21:28.757 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T07:21:28.817 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T07:21:28.913 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-19T07:21:28.913 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T07:21:28.939 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T07:21:29.003 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-19T07:21:29.004 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T07:21:29.029 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T07:21:29.093 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T07:21:29.195 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-19T07:21:29.195 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:21:29.195 INFO:tasks.cephadm:Cluster fsid is c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:21:29.195 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T07:21:29.196 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.159', 'mon.b': '172.21.15.189', 'mon.c': '172.21.15.203'} 2024-09-19T07:21:29.196 INFO:tasks.cephadm:First mon is mon.a on smithi159 2024-09-19T07:21:29.196 INFO:tasks.cephadm:First mgr is x 2024-09-19T07:21:29.196 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T07:21:29.196 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname $(hostname -s) 2024-09-19T07:21:29.256 DEBUG:teuthology.orchestra.run.smithi189:> sudo hostname $(hostname -s) 2024-09-19T07:21:29.282 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname $(hostname -s) 2024-09-19T07:21:29.307 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T07:21:29.308 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:21:29.444 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T07:21:29.586 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:21:29.587 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T07:21:29.588 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T07:21:29.588 DEBUG:teuthology.orchestra.run.smithi159:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T07:21:29.983 INFO:teuthology.orchestra.run.smithi159.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 07:21 /home/ubuntu/cephtest/cephadm 2024-09-19T07:21:29.983 DEBUG:teuthology.orchestra.run.smithi189:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T07:21:30.344 INFO:teuthology.orchestra.run.smithi189.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 07:21 /home/ubuntu/cephtest/cephadm 2024-09-19T07:21:30.344 DEBUG:teuthology.orchestra.run.smithi203:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T07:21:30.723 INFO:teuthology.orchestra.run.smithi203.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 07:21 /home/ubuntu/cephtest/cephadm 2024-09-19T07:21:30.724 DEBUG:teuthology.orchestra.run.smithi159:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T07:21:30.761 DEBUG:teuthology.orchestra.run.smithi189:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T07:21:30.782 DEBUG:teuthology.orchestra.run.smithi203:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T07:21:30.820 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T07:21:30.821 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T07:21:30.825 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T07:21:30.828 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T07:21:32.321 INFO:teuthology.orchestra.run.smithi203.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T07:21:32.355 INFO:teuthology.orchestra.run.smithi189.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T07:21:32.761 INFO:teuthology.orchestra.run.smithi159.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T07:22:13.266 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-19T07:22:13.266 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:22:13.266 INFO:teuthology.orchestra.run.smithi189.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T07:22:13.266 INFO:teuthology.orchestra.run.smithi189.stdout: "repo_digests": [ 2024-09-19T07:22:13.267 INFO:teuthology.orchestra.run.smithi189.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T07:22:13.267 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-19T07:22:13.267 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-19T07:22:17.963 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-09-19T07:22:17.963 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:22:17.964 INFO:teuthology.orchestra.run.smithi203.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T07:22:17.964 INFO:teuthology.orchestra.run.smithi203.stdout: "repo_digests": [ 2024-09-19T07:22:17.964 INFO:teuthology.orchestra.run.smithi203.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T07:22:17.964 INFO:teuthology.orchestra.run.smithi203.stdout: ] 2024-09-19T07:22:17.964 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-09-19T07:22:21.052 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-19T07:22:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:22:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T07:22:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: "repo_digests": [ 2024-09-19T07:22:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T07:22:21.053 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-19T07:22:21.053 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-19T07:22:21.084 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph 2024-09-19T07:22:21.150 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph 2024-09-19T07:22:21.182 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph 2024-09-19T07:22:21.215 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 777 /etc/ceph 2024-09-19T07:22:21.271 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 777 /etc/ceph 2024-09-19T07:22:21.300 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /etc/ceph 2024-09-19T07:22:21.329 INFO:tasks.cephadm:Writing seed config... 2024-09-19T07:22:21.331 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T07:22:21.331 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T07:22:21.331 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T07:22:21.331 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T07:22:21.331 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T07:22:21.331 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-19T07:22:21.331 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T07:22:21.331 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T07:22:21.331 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T07:22:21.331 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-19T07:22:21.331 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-19T07:22:21.331 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-19T07:22:21.332 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-19T07:22:21.332 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T07:22:21.332 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T07:22:21.332 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T07:22:21.332 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T07:22:21.332 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-19T07:22:21.332 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T07:22:21.332 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T07:22:21.332 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T07:22:21.332 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-19T07:22:21.332 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T07:22:21.332 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T07:22:21.332 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T07:22:21.332 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T07:22:21.333 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T07:22:21.333 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T07:22:21.333 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T07:22:21.333 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T07:22:21.333 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T07:22:21.333 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-19T07:22:21.333 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-19T07:22:21.333 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-19T07:22:21.333 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-19T07:22:21.333 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-19T07:22:21.333 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T07:22:21.333 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T07:22:21.333 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-19T07:22:21.333 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-19T07:22:21.334 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-19T07:22:21.334 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-19T07:22:21.334 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-19T07:22:21.334 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-19T07:22:21.334 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T07:22:21.334 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T07:22:21.334 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-19T07:22:21.335 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:22:21.335 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T07:22:21.369 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c9d18048-7657-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T07:22:21.370 DEBUG:teuthology.orchestra.run.smithi159:mon.a> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mon.a.service 2024-09-19T07:22:21.413 DEBUG:teuthology.orchestra.run.smithi159:mgr.x> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mgr.x.service 2024-09-19T07:22:21.458 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T07:22:21.458 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid c9d18048-7657-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.159 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:22:21.718 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-19T07:22:21.718 INFO:teuthology.orchestra.run.smithi159.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', 'c9d18048-7657-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.159', '--skip-admin-label'] 2024-09-19T07:22:21.718 INFO:teuthology.orchestra.run.smithi159.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T07:22:21.719 INFO:teuthology.orchestra.run.smithi159.stdout:Verifying podman|docker is present... 2024-09-19T07:22:21.763 INFO:teuthology.orchestra.run.smithi159.stdout:/bin/podman: stdout 5.2.2 2024-09-19T07:22:21.763 INFO:teuthology.orchestra.run.smithi159.stdout:Verifying lvm2 is present... 2024-09-19T07:22:21.763 INFO:teuthology.orchestra.run.smithi159.stdout:Verifying time synchronization is in place... 2024-09-19T07:22:21.777 INFO:teuthology.orchestra.run.smithi159.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T07:22:21.777 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T07:22:21.789 INFO:teuthology.orchestra.run.smithi159.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T07:22:21.790 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stdout inactive 2024-09-19T07:22:21.803 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stdout enabled 2024-09-19T07:22:21.815 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stdout active 2024-09-19T07:22:21.816 INFO:teuthology.orchestra.run.smithi159.stdout:Unit chronyd.service is enabled and running 2024-09-19T07:22:21.816 INFO:teuthology.orchestra.run.smithi159.stdout:Repeating the final host check... 2024-09-19T07:22:21.867 INFO:teuthology.orchestra.run.smithi159.stdout:/bin/podman: stdout 5.2.2 2024-09-19T07:22:21.867 INFO:teuthology.orchestra.run.smithi159.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T07:22:21.867 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl is present 2024-09-19T07:22:21.867 INFO:teuthology.orchestra.run.smithi159.stdout:lvcreate is present 2024-09-19T07:22:21.880 INFO:teuthology.orchestra.run.smithi159.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T07:22:21.880 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T07:22:21.892 INFO:teuthology.orchestra.run.smithi159.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T07:22:21.892 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stdout inactive 2024-09-19T07:22:21.906 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stdout enabled 2024-09-19T07:22:21.918 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stdout active 2024-09-19T07:22:21.919 INFO:teuthology.orchestra.run.smithi159.stdout:Unit chronyd.service is enabled and running 2024-09-19T07:22:21.919 INFO:teuthology.orchestra.run.smithi159.stdout:Host looks OK 2024-09-19T07:22:21.919 INFO:teuthology.orchestra.run.smithi159.stdout:Cluster fsid: c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:22:21.919 INFO:teuthology.orchestra.run.smithi159.stdout:Acquiring lock 140269622326272 on /run/cephadm/c9d18048-7657-11ef-bceb-c7b262605968.lock 2024-09-19T07:22:21.919 INFO:teuthology.orchestra.run.smithi159.stdout:Lock 140269622326272 acquired on /run/cephadm/c9d18048-7657-11ef-bceb-c7b262605968.lock 2024-09-19T07:22:21.920 INFO:teuthology.orchestra.run.smithi159.stdout:Verifying IP 172.21.15.159 port 3300 ... 2024-09-19T07:22:21.920 INFO:teuthology.orchestra.run.smithi159.stdout:Verifying IP 172.21.15.159 port 6789 ... 2024-09-19T07:22:21.920 INFO:teuthology.orchestra.run.smithi159.stdout:Base mon IP(s) is [172.21.15.159:3300, 172.21.15.159:6789], mon addrv is [v2:172.21.15.159:3300,v1:172.21.15.159:6789] 2024-09-19T07:22:21.927 INFO:teuthology.orchestra.run.smithi159.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.159 metric 100 2024-09-19T07:22:21.927 INFO:teuthology.orchestra.run.smithi159.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.159 metric 100 2024-09-19T07:22:21.933 INFO:teuthology.orchestra.run.smithi159.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T07:22:21.933 INFO:teuthology.orchestra.run.smithi159.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-19T07:22:21.933 INFO:teuthology.orchestra.run.smithi159.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-19T07:22:21.938 INFO:teuthology.orchestra.run.smithi159.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T07:22:21.938 INFO:teuthology.orchestra.run.smithi159.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T07:22:21.938 INFO:teuthology.orchestra.run.smithi159.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T07:22:21.939 INFO:teuthology.orchestra.run.smithi159.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-19T07:22:21.939 INFO:teuthology.orchestra.run.smithi159.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:724f/64 scope link noprefixroute 2024-09-19T07:22:21.939 INFO:teuthology.orchestra.run.smithi159.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T07:22:21.939 INFO:teuthology.orchestra.run.smithi159.stdout:Mon IP `172.21.15.159` is in CIDR network `172.21.0.0/20` 2024-09-19T07:22:21.939 INFO:teuthology.orchestra.run.smithi159.stdout:Mon IP `172.21.15.159` is in CIDR network `172.21.0.0/20` 2024-09-19T07:22:21.939 INFO:teuthology.orchestra.run.smithi159.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T07:22:21.940 INFO:teuthology.orchestra.run.smithi159.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T07:22:21.941 INFO:teuthology.orchestra.run.smithi159.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T07:22:22.422 INFO:teuthology.orchestra.run.smithi159.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T07:22:22.422 INFO:teuthology.orchestra.run.smithi159.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T07:22:22.423 INFO:teuthology.orchestra.run.smithi159.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T07:22:22.423 INFO:teuthology.orchestra.run.smithi159.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T07:22:22.423 INFO:teuthology.orchestra.run.smithi159.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T07:22:22.423 INFO:teuthology.orchestra.run.smithi159.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T07:22:22.423 INFO:teuthology.orchestra.run.smithi159.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T07:22:23.492 INFO:teuthology.orchestra.run.smithi159.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T07:22:23.493 INFO:teuthology.orchestra.run.smithi159.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T07:22:23.493 INFO:teuthology.orchestra.run.smithi159.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T07:22:24.533 INFO:teuthology.orchestra.run.smithi159.stdout:stat: stdout 167 167 2024-09-19T07:22:24.533 INFO:teuthology.orchestra.run.smithi159.stdout:Creating initial keys... 2024-09-19T07:22:25.670 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph-authtool: stdout AQAw0etmnUMLORAAhArzjzp28TkduMEBqmzlYg== 2024-09-19T07:22:26.829 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph-authtool: stdout AQAy0etmnDg1AhAAkxvXlZsRzs5JqRYCKyN2pg== 2024-09-19T07:22:27.891 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph-authtool: stdout AQAz0etm+/b7DBAAcflLS/Bks8Dnlbc81QS7Pw== 2024-09-19T07:22:27.892 INFO:teuthology.orchestra.run.smithi159.stdout:Creating initial monmap... 2024-09-19T07:22:29.005 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T07:22:29.005 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T07:22:29.005 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:22:29.005 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T07:22:29.005 INFO:teuthology.orchestra.run.smithi159.stdout:monmaptool for a [v2:172.21.15.159:3300,v1:172.21.15.159:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T07:22:29.005 INFO:teuthology.orchestra.run.smithi159.stdout:setting min_mon_release = quincy 2024-09-19T07:22:29.005 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/monmaptool: set fsid to c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:22:29.005 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T07:22:29.005 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:22:29.005 INFO:teuthology.orchestra.run.smithi159.stdout:Creating mon... 2024-09-19T07:22:30.036 INFO:teuthology.orchestra.run.smithi159.stdout:create mon.a on 2024-09-19T07:22:30.341 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T07:22:30.566 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T07:22:30.828 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c9d18048-7657-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-c9d18048-7657-11ef-bceb-c7b262605968.target. 2024-09-19T07:22:30.829 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c9d18048-7657-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-c9d18048-7657-11ef-bceb-c7b262605968.target. 2024-09-19T07:22:31.144 INFO:teuthology.orchestra.run.smithi159.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c9d18048-7657-11ef-bceb-c7b262605968@mon.a 2024-09-19T07:22:31.144 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c9d18048-7657-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-c9d18048-7657-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-19T07:22:31.412 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c9d18048-7657-11ef-bceb-c7b262605968.target.wants/ceph-c9d18048-7657-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-c9d18048-7657-11ef-bceb-c7b262605968@.service. 2024-09-19T07:22:32.062 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:31 smithi159 podman[30095]: 2024-09-19 07:22:31.999511275 +0000 UTC m=+0.373540943 container start 72505973366cd2d2d6b84bab8b868690eaca985fe4166df14af244adb9d2e375 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T07:22:32.111 INFO:teuthology.orchestra.run.smithi159.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T07:22:32.111 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stdout disabled 2024-09-19T07:22:32.124 INFO:teuthology.orchestra.run.smithi159.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T07:22:32.124 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stdout inactive 2024-09-19T07:22:32.124 INFO:teuthology.orchestra.run.smithi159.stdout:firewalld.service is not enabled 2024-09-19T07:22:32.124 INFO:teuthology.orchestra.run.smithi159.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T07:22:32.125 INFO:teuthology.orchestra.run.smithi159.stdout:Waiting for mon to start... 2024-09-19T07:22:32.125 INFO:teuthology.orchestra.run.smithi159.stdout:Waiting for mon... 2024-09-19T07:22:32.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:32 smithi159 ceph-mon[30114]: mkfs c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:22:32.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:32 smithi159 ceph-mon[30114]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T07:22:32.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:32 smithi159 bash[30095]: 72505973366cd2d2d6b84bab8b868690eaca985fe4166df14af244adb9d2e375 2024-09-19T07:22:32.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:32 smithi159 systemd[1]: Started Ceph mon.a for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:22:33.308 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:33 smithi159 ceph-mon[30114]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T07:22:33.308 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:33 smithi159 ceph-mon[30114]: monmap epoch 1 2024-09-19T07:22:33.308 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:33 smithi159 ceph-mon[30114]: fsid c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:22:33.308 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:33 smithi159 ceph-mon[30114]: last_changed 2024-09-19T07:22:28.312056+0000 2024-09-19T07:22:33.308 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:33 smithi159 ceph-mon[30114]: created 2024-09-19T07:22:28.312056+0000 2024-09-19T07:22:33.308 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:33 smithi159 ceph-mon[30114]: min_mon_release 19 (squid) 2024-09-19T07:22:33.308 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:33 smithi159 ceph-mon[30114]: election_strategy: 1 2024-09-19T07:22:33.309 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:33 smithi159 ceph-mon[30114]: 0: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.a 2024-09-19T07:22:33.309 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:33 smithi159 ceph-mon[30114]: fsmap 2024-09-19T07:22:33.309 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:33 smithi159 ceph-mon[30114]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T07:22:33.309 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:33 smithi159 ceph-mon[30114]: mgrmap e1: no daemons active 2024-09-19T07:22:33.309 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:33 smithi159 ceph-mon[30114]: from='client.? 172.21.15.159:0/4254117481' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T07:22:33.312 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T07:22:33.320 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout id: c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:22:33.320 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T07:22:33.320 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout 2024-09-19T07:22:33.320 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout services: 2024-09-19T07:22:33.321 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.643817s) 2024-09-19T07:22:33.321 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T07:22:33.321 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T07:22:33.321 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout 2024-09-19T07:22:33.321 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout data: 2024-09-19T07:22:33.321 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T07:22:33.321 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T07:22:33.322 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T07:22:33.322 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T07:22:33.322 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout 2024-09-19T07:22:33.322 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.626+0000 7f6cf4a00640 1 Processor -- start 2024-09-19T07:22:33.322 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.626+0000 7f6cf4a00640 1 -- start start 2024-09-19T07:22:33.322 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.626+0000 7f6cf4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf0109920 0x7f6cf0109d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:33.322 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.626+0000 7f6cf4a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf010a310 con 0x7f6cf0109920 2024-09-19T07:22:33.323 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.627+0000 7f6cede00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf0109920 0x7f6cf0109d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:33.323 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.627+0000 7f6cede00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf0109920 0x7f6cf0109d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42386/0 (socket says 172.21.15.159:42386) 2024-09-19T07:22:33.323 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.627+0000 7f6cede00640 1 -- 172.21.15.159:0/689079266 learned_addr learned my addr 172.21.15.159:0/689079266 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:33.323 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.627+0000 7f6cede00640 1 -- 172.21.15.159:0/689079266 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf010aa10 con 0x7f6cf0109920 2024-09-19T07:22:33.323 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.630+0000 7f6cede00640 1 --2- 172.21.15.159:0/689079266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf0109920 0x7f6cf0109d40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009820 tx=0x7f6ce002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74aed8263327ee90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:33.323 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.630+0000 7f6ceca00640 1 -- 172.21.15.159:0/689079266 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ce0037070 con 0x7f6cf0109920 2024-09-19T07:22:33.323 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.630+0000 7f6ceca00640 1 -- 172.21.15.159:0/689079266 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce0032440 con 0x7f6cf0109920 2024-09-19T07:22:33.323 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.630+0000 7f6ceca00640 1 -- 172.21.15.159:0/689079266 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ce0032740 con 0x7f6cf0109920 2024-09-19T07:22:33.323 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.631+0000 7f6cf4a00640 1 -- 172.21.15.159:0/689079266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf0109920 msgr2=0x7f6cf0109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:33.323 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.631+0000 7f6cf4a00640 1 --2- 172.21.15.159:0/689079266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf0109920 0x7f6cf0109d40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009820 tx=0x7f6ce002a270 comp rx=0 tx=0).stop 2024-09-19T07:22:33.323 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.631+0000 7f6cf4a00640 1 -- 172.21.15.159:0/689079266 shutdown_connections 2024-09-19T07:22:33.324 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.631+0000 7f6cf4a00640 1 --2- 172.21.15.159:0/689079266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf0109920 0x7f6cf0109d40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:33.324 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.631+0000 7f6cf4a00640 1 -- 172.21.15.159:0/689079266 >> 172.21.15.159:0/689079266 conn(0x7f6cf007cf40 msgr2=0x7f6cf007d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:33.324 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.631+0000 7f6cf4a00640 1 -- 172.21.15.159:0/689079266 shutdown_connections 2024-09-19T07:22:33.324 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.631+0000 7f6cf4a00640 1 -- 172.21.15.159:0/689079266 wait complete. 2024-09-19T07:22:33.324 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.632+0000 7f6cf4a00640 1 Processor -- start 2024-09-19T07:22:33.324 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.632+0000 7f6cf4a00640 1 -- start start 2024-09-19T07:22:33.324 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.632+0000 7f6cf4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf0109920 0x7f6cf019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:33.324 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.632+0000 7f6cf4a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf010aee0 con 0x7f6cf0109920 2024-09-19T07:22:33.324 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.633+0000 7f6cede00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf0109920 0x7f6cf019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:33.324 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.633+0000 7f6cede00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf0109920 0x7f6cf019cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42396/0 (socket says 172.21.15.159:42396) 2024-09-19T07:22:33.325 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.633+0000 7f6cede00640 1 -- 172.21.15.159:0/4254117481 learned_addr learned my addr 172.21.15.159:0/4254117481 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:33.325 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.633+0000 7f6cede00640 1 -- 172.21.15.159:0/4254117481 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf019d510 con 0x7f6cf0109920 2024-09-19T07:22:33.325 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.633+0000 7f6cede00640 1 --2- 172.21.15.159:0/4254117481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf0109920 0x7f6cf019cfd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00090b0 tx=0x7f6ce00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:33.325 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.634+0000 7f6ce6a00640 1 -- 172.21.15.159:0/4254117481 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ce0041070 con 0x7f6cf0109920 2024-09-19T07:22:33.325 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.634+0000 7f6ce6a00640 1 -- 172.21.15.159:0/4254117481 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce003c7a0 con 0x7f6cf0109920 2024-09-19T07:22:33.325 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.634+0000 7f6ce6a00640 1 -- 172.21.15.159:0/4254117481 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ce0037040 con 0x7f6cf0109920 2024-09-19T07:22:33.325 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.634+0000 7f6cf4a00640 1 -- 172.21.15.159:0/4254117481 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf019d7a0 con 0x7f6cf0109920 2024-09-19T07:22:33.325 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.634+0000 7f6cf4a00640 1 -- 172.21.15.159:0/4254117481 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf01a0490 con 0x7f6cf0109920 2024-09-19T07:22:33.325 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.634+0000 7f6ce6a00640 1 -- 172.21.15.159:0/4254117481 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6ce0044070 con 0x7f6cf0109920 2024-09-19T07:22:33.326 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.634+0000 7f6ce6a00640 1 -- 172.21.15.159:0/4254117481 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6ce003d3f0 con 0x7f6cf0109920 2024-09-19T07:22:33.326 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.635+0000 7f6cf4a00640 1 -- 172.21.15.159:0/4254117481 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf010b580 con 0x7f6cf0109920 2024-09-19T07:22:33.326 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.638+0000 7f6ce6a00640 1 -- 172.21.15.159:0/4254117481 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6ce0046450 con 0x7f6cf0109920 2024-09-19T07:22:33.326 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.700+0000 7f6cf4a00640 1 -- 172.21.15.159:0/4254117481 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f6cf01a09e0 con 0x7f6cf0109920 2024-09-19T07:22:33.326 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.701+0000 7f6ce6a00640 1 -- 172.21.15.159:0/4254117481 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f6ce0046630 con 0x7f6cf0109920 2024-09-19T07:22:33.326 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.702+0000 7f6cf4a00640 1 -- 172.21.15.159:0/4254117481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf0109920 msgr2=0x7f6cf019cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:33.326 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.702+0000 7f6cf4a00640 1 --2- 172.21.15.159:0/4254117481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf0109920 0x7f6cf019cfd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00090b0 tx=0x7f6ce00046c0 comp rx=0 tx=0).stop 2024-09-19T07:22:33.327 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.702+0000 7f6cf4a00640 1 -- 172.21.15.159:0/4254117481 shutdown_connections 2024-09-19T07:22:33.327 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.702+0000 7f6cf4a00640 1 --2- 172.21.15.159:0/4254117481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf0109920 0x7f6cf019cfd0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:33.327 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.702+0000 7f6cf4a00640 1 -- 172.21.15.159:0/4254117481 >> 172.21.15.159:0/4254117481 conn(0x7f6cf007cf40 msgr2=0x7f6cf01912c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:33.327 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.702+0000 7f6cf4a00640 1 -- 172.21.15.159:0/4254117481 shutdown_connections 2024-09-19T07:22:33.327 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:32.702+0000 7f6cf4a00640 1 -- 172.21.15.159:0/4254117481 wait complete. 2024-09-19T07:22:33.327 INFO:teuthology.orchestra.run.smithi159.stdout:mon is available 2024-09-19T07:22:33.327 INFO:teuthology.orchestra.run.smithi159.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T07:22:34.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:34 smithi159 ceph-mon[30114]: from='client.? 172.21.15.159:0/2453021884' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T07:22:34.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:34 smithi159 ceph-mon[30114]: from='client.? 172.21.15.159:0/2453021884' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T07:22:34.500 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout 2024-09-19T07:22:34.500 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T07:22:34.501 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout fsid = c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:22:34.501 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.159:3300,v1:172.21.15.159:6789] 2024-09-19T07:22:34.501 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T07:22:34.501 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T07:22:34.501 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T07:22:34.501 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T07:22:34.501 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout 2024-09-19T07:22:34.501 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T07:22:34.501 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T07:22:34.501 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout 2024-09-19T07:22:34.501 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T07:22:34.501 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T07:22:34.501 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T07:22:34.502 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T07:22:34.502 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T07:22:34.502 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T07:22:34.502 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.785+0000 7fdc10c00640 1 Processor -- start 2024-09-19T07:22:34.502 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.785+0000 7fdc10c00640 1 -- start start 2024-09-19T07:22:34.502 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.786+0000 7fdc10c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc0c107680 0x7fdc0c107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:34.502 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.786+0000 7fdc10c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c107fc0 con 0x7fdc0c107680 2024-09-19T07:22:34.502 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.786+0000 7fdc09e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc0c107680 0x7fdc0c107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:34.503 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.787+0000 7fdc09e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc0c107680 0x7fdc0c107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42412/0 (socket says 172.21.15.159:42412) 2024-09-19T07:22:34.503 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.787+0000 7fdc09e00640 1 -- 172.21.15.159:0/315179634 learned_addr learned my addr 172.21.15.159:0/315179634 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:34.503 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.787+0000 7fdc09e00640 1 -- 172.21.15.159:0/315179634 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0c108140 con 0x7fdc0c107680 2024-09-19T07:22:34.503 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.788+0000 7fdc09e00640 1 --2- 172.21.15.159:0/315179634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc0c107680 0x7fdc0c107a80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009820 tx=0x7fdbfc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef744c813ac7e80b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:34.503 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.788+0000 7fdc09400640 1 -- 172.21.15.159:0/315179634 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbfc037070 con 0x7fdc0c107680 2024-09-19T07:22:34.503 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.788+0000 7fdc09400640 1 -- 172.21.15.159:0/315179634 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbfc032440 con 0x7fdc0c107680 2024-09-19T07:22:34.503 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.788+0000 7fdc09400640 1 -- 172.21.15.159:0/315179634 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbfc032740 con 0x7fdc0c107680 2024-09-19T07:22:34.504 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.789+0000 7fdc10c00640 1 -- 172.21.15.159:0/315179634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc0c107680 msgr2=0x7fdc0c107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:34.504 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.789+0000 7fdc10c00640 1 --2- 172.21.15.159:0/315179634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc0c107680 0x7fdc0c107a80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009820 tx=0x7fdbfc02a270 comp rx=0 tx=0).stop 2024-09-19T07:22:34.504 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.789+0000 7fdc10c00640 1 -- 172.21.15.159:0/315179634 shutdown_connections 2024-09-19T07:22:34.504 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.789+0000 7fdc10c00640 1 --2- 172.21.15.159:0/315179634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc0c107680 0x7fdc0c107a80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:34.504 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.789+0000 7fdc10c00640 1 -- 172.21.15.159:0/315179634 >> 172.21.15.159:0/315179634 conn(0x7fdc0c102eb0 msgr2=0x7fdc0c1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:34.504 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.789+0000 7fdc10c00640 1 -- 172.21.15.159:0/315179634 shutdown_connections 2024-09-19T07:22:34.504 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.789+0000 7fdc10c00640 1 -- 172.21.15.159:0/315179634 wait complete. 2024-09-19T07:22:34.504 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.790+0000 7fdc10c00640 1 Processor -- start 2024-09-19T07:22:34.504 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.790+0000 7fdc10c00640 1 -- start start 2024-09-19T07:22:34.504 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.790+0000 7fdc10c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc0c107680 0x7fdc0c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:34.504 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.790+0000 7fdc10c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c108ba0 con 0x7fdc0c107680 2024-09-19T07:22:34.505 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.791+0000 7fdc09e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc0c107680 0x7fdc0c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:34.505 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.791+0000 7fdc09e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc0c107680 0x7fdc0c19ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42426/0 (socket says 172.21.15.159:42426) 2024-09-19T07:22:34.505 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.791+0000 7fdc09e00640 1 -- 172.21.15.159:0/2453021884 learned_addr learned my addr 172.21.15.159:0/2453021884 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:34.505 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.791+0000 7fdc09e00640 1 -- 172.21.15.159:0/2453021884 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0c19b350 con 0x7fdc0c107680 2024-09-19T07:22:34.505 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.791+0000 7fdc09e00640 1 --2- 172.21.15.159:0/2453021884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc0c107680 0x7fdc0c19ae10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0090b0 tx=0x7fdbfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:34.505 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.791+0000 7fdc02a00640 1 -- 172.21.15.159:0/2453021884 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbfc041070 con 0x7fdc0c107680 2024-09-19T07:22:34.505 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.792+0000 7fdc02a00640 1 -- 172.21.15.159:0/2453021884 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbfc03c7a0 con 0x7fdc0c107680 2024-09-19T07:22:34.505 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.792+0000 7fdc10c00640 1 -- 172.21.15.159:0/2453021884 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0c19b5e0 con 0x7fdc0c107680 2024-09-19T07:22:34.505 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.792+0000 7fdc02a00640 1 -- 172.21.15.159:0/2453021884 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbfc037040 con 0x7fdc0c107680 2024-09-19T07:22:34.505 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.792+0000 7fdc10c00640 1 -- 172.21.15.159:0/2453021884 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc0c19e2d0 con 0x7fdc0c107680 2024-09-19T07:22:34.506 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.793+0000 7fdc02a00640 1 -- 172.21.15.159:0/2453021884 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdbfc044070 con 0x7fdc0c107680 2024-09-19T07:22:34.506 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.793+0000 7fdc02a00640 1 -- 172.21.15.159:0/2453021884 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdbfc03d3f0 con 0x7fdc0c107680 2024-09-19T07:22:34.506 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.793+0000 7fdc10c00640 1 -- 172.21.15.159:0/2453021884 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc0c107b00 con 0x7fdc0c107680 2024-09-19T07:22:34.506 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.795+0000 7fdc02a00640 1 -- 172.21.15.159:0/2453021884 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fdbfc046450 con 0x7fdc0c107680 2024-09-19T07:22:34.506 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.855+0000 7fdc10c00640 1 -- 172.21.15.159:0/2453021884 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fdc0c19bac0 con 0x7fdc0c107680 2024-09-19T07:22:34.506 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.860+0000 7fdc02a00640 1 -- 172.21.15.159:0/2453021884 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7fdc0c19bac0 con 0x7fdc0c107680 2024-09-19T07:22:34.506 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.861+0000 7fdc02a00640 1 -- 172.21.15.159:0/2453021884 <== mon.0 v2:172.21.15.159:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdbfc056070 con 0x7fdc0c107680 2024-09-19T07:22:34.506 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.861+0000 7fdc10c00640 1 -- 172.21.15.159:0/2453021884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc0c107680 msgr2=0x7fdc0c19ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:34.506 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.861+0000 7fdc10c00640 1 --2- 172.21.15.159:0/2453021884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc0c107680 0x7fdc0c19ae10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0090b0 tx=0x7fdbfc0046c0 comp rx=0 tx=0).stop 2024-09-19T07:22:34.506 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.862+0000 7fdc10c00640 1 -- 172.21.15.159:0/2453021884 shutdown_connections 2024-09-19T07:22:34.506 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.862+0000 7fdc10c00640 1 --2- 172.21.15.159:0/2453021884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc0c107680 0x7fdc0c19ae10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:34.506 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.862+0000 7fdc10c00640 1 -- 172.21.15.159:0/2453021884 >> 172.21.15.159:0/2453021884 conn(0x7fdc0c102eb0 msgr2=0x7fdc0c18f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:34.507 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.862+0000 7fdc10c00640 1 -- 172.21.15.159:0/2453021884 shutdown_connections 2024-09-19T07:22:34.507 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:33.862+0000 7fdc10c00640 1 -- 172.21.15.159:0/2453021884 wait complete. 2024-09-19T07:22:34.507 INFO:teuthology.orchestra.run.smithi159.stdout:Generating new minimal ceph.conf... 2024-09-19T07:22:35.360 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:35 smithi159 ceph-mon[30114]: from='client.? 172.21.15.159:0/30483575' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:22:35.618 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.963+0000 7f9e84a00640 1 Processor -- start 2024-09-19T07:22:35.618 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.963+0000 7f9e84a00640 1 -- start start 2024-09-19T07:22:35.618 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.963+0000 7f9e84a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e8007ddb0 0x7f9e8007c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:35.618 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.963+0000 7f9e84a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8007c860 con 0x7f9e8007ddb0 2024-09-19T07:22:35.618 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.964+0000 7f9e7de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e8007ddb0 0x7f9e8007c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:35.618 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.964+0000 7f9e7de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e8007ddb0 0x7f9e8007c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42434/0 (socket says 172.21.15.159:42434) 2024-09-19T07:22:35.618 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.964+0000 7f9e7de00640 1 -- 172.21.15.159:0/1563653300 learned_addr learned my addr 172.21.15.159:0/1563653300 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:35.618 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.964+0000 7f9e7de00640 1 -- 172.21.15.159:0/1563653300 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8007c9e0 con 0x7f9e8007ddb0 2024-09-19T07:22:35.618 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.965+0000 7f9e7de00640 1 --2- 172.21.15.159:0/1563653300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e8007ddb0 0x7f9e8007c320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009820 tx=0x7f9e7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e939fb301667c69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:35.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.965+0000 7f9e7ca00640 1 -- 172.21.15.159:0/1563653300 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e70037070 con 0x7f9e8007ddb0 2024-09-19T07:22:35.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.965+0000 7f9e7ca00640 1 -- 172.21.15.159:0/1563653300 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9e70032900 con 0x7f9e8007ddb0 2024-09-19T07:22:35.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.965+0000 7f9e7ca00640 1 -- 172.21.15.159:0/1563653300 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e70030850 con 0x7f9e8007ddb0 2024-09-19T07:22:35.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.966+0000 7f9e84a00640 1 -- 172.21.15.159:0/1563653300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e8007ddb0 msgr2=0x7f9e8007c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:35.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.966+0000 7f9e84a00640 1 --2- 172.21.15.159:0/1563653300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e8007ddb0 0x7f9e8007c320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009820 tx=0x7f9e7002a270 comp rx=0 tx=0).stop 2024-09-19T07:22:35.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.966+0000 7f9e84a00640 1 -- 172.21.15.159:0/1563653300 shutdown_connections 2024-09-19T07:22:35.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.966+0000 7f9e84a00640 1 --2- 172.21.15.159:0/1563653300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e8007ddb0 0x7f9e8007c320 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:35.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.966+0000 7f9e84a00640 1 -- 172.21.15.159:0/1563653300 >> 172.21.15.159:0/1563653300 conn(0x7f9e80102eb0 msgr2=0x7f9e801052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:35.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.966+0000 7f9e84a00640 1 -- 172.21.15.159:0/1563653300 shutdown_connections 2024-09-19T07:22:35.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.966+0000 7f9e84a00640 1 -- 172.21.15.159:0/1563653300 wait complete. 2024-09-19T07:22:35.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.967+0000 7f9e84a00640 1 Processor -- start 2024-09-19T07:22:35.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.967+0000 7f9e84a00640 1 -- start start 2024-09-19T07:22:35.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.967+0000 7f9e84a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e8007ddb0 0x7f9e8019d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:35.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.968+0000 7f9e84a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8010dbe0 con 0x7f9e8007ddb0 2024-09-19T07:22:35.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.968+0000 7f9e7de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e8007ddb0 0x7f9e8019d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:35.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.968+0000 7f9e7de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e8007ddb0 0x7f9e8019d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42450/0 (socket says 172.21.15.159:42450) 2024-09-19T07:22:35.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.968+0000 7f9e7de00640 1 -- 172.21.15.159:0/30483575 learned_addr learned my addr 172.21.15.159:0/30483575 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:35.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.968+0000 7f9e7de00640 1 -- 172.21.15.159:0/30483575 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8019ddd0 con 0x7f9e8007ddb0 2024-09-19T07:22:35.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.968+0000 7f9e7de00640 1 --2- 172.21.15.159:0/30483575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e8007ddb0 0x7f9e8019d890 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9e700097f0 tx=0x7f9e70030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:35.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.969+0000 7f9e76a00640 1 -- 172.21.15.159:0/30483575 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e70042070 con 0x7f9e8007ddb0 2024-09-19T07:22:35.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.969+0000 7f9e76a00640 1 -- 172.21.15.159:0/30483575 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9e70032ca0 con 0x7f9e8007ddb0 2024-09-19T07:22:35.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.969+0000 7f9e76a00640 1 -- 172.21.15.159:0/30483575 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e70037040 con 0x7f9e8007ddb0 2024-09-19T07:22:35.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.969+0000 7f9e84a00640 1 -- 172.21.15.159:0/30483575 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8019e060 con 0x7f9e8007ddb0 2024-09-19T07:22:35.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.969+0000 7f9e84a00640 1 -- 172.21.15.159:0/30483575 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e801a0d50 con 0x7f9e8007ddb0 2024-09-19T07:22:35.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.969+0000 7f9e76a00640 1 -- 172.21.15.159:0/30483575 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9e70045070 con 0x7f9e8007ddb0 2024-09-19T07:22:35.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.969+0000 7f9e76a00640 1 -- 172.21.15.159:0/30483575 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9e70047a90 con 0x7f9e8007ddb0 2024-09-19T07:22:35.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.970+0000 7f9e84a00640 1 -- 172.21.15.159:0/30483575 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e8007c3a0 con 0x7f9e8007ddb0 2024-09-19T07:22:35.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:34.973+0000 7f9e76a00640 1 -- 172.21.15.159:0/30483575 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9e7003db10 con 0x7f9e8007ddb0 2024-09-19T07:22:35.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:35.033+0000 7f9e84a00640 1 -- 172.21.15.159:0/30483575 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f9e801a12d0 con 0x7f9e8007ddb0 2024-09-19T07:22:35.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:35.034+0000 7f9e76a00640 1 -- 172.21.15.159:0/30483575 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f9e7003d430 con 0x7f9e8007ddb0 2024-09-19T07:22:35.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:35.034+0000 7f9e84a00640 1 -- 172.21.15.159:0/30483575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e8007ddb0 msgr2=0x7f9e8019d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:35.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:35.034+0000 7f9e84a00640 1 --2- 172.21.15.159:0/30483575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e8007ddb0 0x7f9e8019d890 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9e700097f0 tx=0x7f9e70030ec0 comp rx=0 tx=0).stop 2024-09-19T07:22:35.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:35.035+0000 7f9e84a00640 1 -- 172.21.15.159:0/30483575 shutdown_connections 2024-09-19T07:22:35.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:35.035+0000 7f9e84a00640 1 --2- 172.21.15.159:0/30483575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e8007ddb0 0x7f9e8019d890 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:35.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:35.035+0000 7f9e84a00640 1 -- 172.21.15.159:0/30483575 >> 172.21.15.159:0/30483575 conn(0x7f9e80102eb0 msgr2=0x7f9e80103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:35.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:35.035+0000 7f9e84a00640 1 -- 172.21.15.159:0/30483575 shutdown_connections 2024-09-19T07:22:35.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:35.035+0000 7f9e84a00640 1 -- 172.21.15.159:0/30483575 wait complete. 2024-09-19T07:22:35.621 INFO:teuthology.orchestra.run.smithi159.stdout:Restarting the monitor... 2024-09-19T07:22:35.987 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:35 smithi159 systemd[1]: Stopping Ceph mon.a for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:22:36.317 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:35 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-a[30110]: 2024-09-19T07:22:35.985+0000 7fa2f8800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:22:36.317 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:35 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-a[30110]: 2024-09-19T07:22:35.985+0000 7fa2f8800640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T07:22:36.317 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:36 smithi159 podman[30350]: 2024-09-19 07:22:36.0070825 +0000 UTC m=+0.168438546 container died 72505973366cd2d2d6b84bab8b868690eaca985fe4166df14af244adb9d2e375 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T07:22:36.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:36 smithi159 podman[30350]: 2024-09-19 07:22:36.440625164 +0000 UTC m=+0.601981211 container cleanup 72505973366cd2d2d6b84bab8b868690eaca985fe4166df14af244adb9d2e375 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True) 2024-09-19T07:22:36.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:36 smithi159 bash[30350]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-a 2024-09-19T07:22:36.946 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:36 smithi159 podman[30362]: 2024-09-19 07:22:36.683344559 +0000 UTC m=+0.661984244 container remove 72505973366cd2d2d6b84bab8b868690eaca985fe4166df14af244adb9d2e375 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-19T07:22:37.287 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:36 smithi159 systemd[1]: ceph-c9d18048-7657-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T07:22:37.288 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:36 smithi159 systemd[1]: Stopped Ceph mon.a for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:22:37.288 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:36 smithi159 systemd[1]: ceph-c9d18048-7657-11ef-bceb-c7b262605968@mon.a.service: Consumed 1.203s CPU time. 2024-09-19T07:22:37.288 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:36 smithi159 systemd[1]: Starting Ceph mon.a for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:22:37.543 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 podman[30453]: 2024-09-19 07:22:37.192121772 +0000 UTC m=+0.026485977 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:22:37.543 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 podman[30453]: 2024-09-19 07:22:37.334482127 +0000 UTC m=+0.168846320 container create 37afdc5581e3e9dde80d4424cafe97f73df3462a265d8296a8ddb3d59549ac06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:22:37.543 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 podman[30453]: 2024-09-19 07:22:37.536730893 +0000 UTC m=+0.371095086 container init 37afdc5581e3e9dde80d4424cafe97f73df3462a265d8296a8ddb3d59549ac06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T07:22:37.543 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 podman[30453]: 2024-09-19 07:22:37.543301844 +0000 UTC m=+0.377666037 container start 37afdc5581e3e9dde80d4424cafe97f73df3462a265d8296a8ddb3d59549ac06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-a, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:22:37.669 INFO:teuthology.orchestra.run.smithi159.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T07:22:37.810 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T07:22:37.810 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T07:22:37.810 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: pidfile_write: ignore empty --pid-file 2024-09-19T07:22:37.810 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: load: jerasure load: lrc 2024-09-19T07:22:37.810 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: RocksDB version: 7.9.2 2024-09-19T07:22:37.810 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Git sha 0 2024-09-19T07:22:37.811 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T07:22:37.811 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: DB SUMMARY 2024-09-19T07:22:37.811 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: DB Session ID: YXYLXT1C4J9NHBJFECH9 2024-09-19T07:22:37.811 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: CURRENT file: CURRENT 2024-09-19T07:22:37.811 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T07:22:37.811 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T07:22:37.811 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T07:22:37.811 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105386 ; 2024-09-19T07:22:37.811 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.error_if_exists: 0 2024-09-19T07:22:37.811 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.create_if_missing: 0 2024-09-19T07:22:37.811 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.paranoid_checks: 1 2024-09-19T07:22:37.811 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T07:22:37.812 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T07:22:37.812 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T07:22:37.812 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.env: 0x5614e0ad2940 2024-09-19T07:22:37.812 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T07:22:37.812 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.info_log: 0x5615148d96a0 2024-09-19T07:22:37.812 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T07:22:37.812 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.statistics: (nil) 2024-09-19T07:22:37.812 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.use_fsync: 0 2024-09-19T07:22:37.812 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_log_file_size: 0 2024-09-19T07:22:37.812 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T07:22:37.812 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T07:22:37.812 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T07:22:37.813 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T07:22:37.813 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.allow_fallocate: 1 2024-09-19T07:22:37.813 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T07:22:37.813 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T07:22:37.813 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.use_direct_reads: 0 2024-09-19T07:22:37.813 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T07:22:37.813 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T07:22:37.813 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.db_log_dir: 2024-09-19T07:22:37.813 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.wal_dir: 2024-09-19T07:22:37.813 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T07:22:37.813 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T07:22:37.813 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T07:22:37.813 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T07:22:37.814 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T07:22:37.814 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T07:22:37.814 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T07:22:37.814 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T07:22:37.814 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.write_buffer_manager: 0x5615148e8dc0 2024-09-19T07:22:37.814 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T07:22:37.814 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T07:22:37.814 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T07:22:37.814 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T07:22:37.814 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T07:22:37.814 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T07:22:37.814 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T07:22:37.814 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T07:22:37.815 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.unordered_write: 0 2024-09-19T07:22:37.815 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T07:22:37.815 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T07:22:37.815 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T07:22:37.815 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T07:22:37.815 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.row_cache: None 2024-09-19T07:22:37.815 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.wal_filter: None 2024-09-19T07:22:37.815 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T07:22:37.815 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T07:22:37.815 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.two_write_queues: 0 2024-09-19T07:22:37.815 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T07:22:37.815 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.wal_compression: 0 2024-09-19T07:22:37.815 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.atomic_flush: 0 2024-09-19T07:22:37.815 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T07:22:37.816 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T07:22:37.816 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T07:22:37.816 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.log_readahead_size: 0 2024-09-19T07:22:37.816 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T07:22:37.816 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T07:22:37.816 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T07:22:37.816 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T07:22:37.816 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T07:22:37.816 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T07:22:37.816 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T07:22:37.817 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_background_jobs: 2 2024-09-19T07:22:37.818 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_background_compactions: -1 2024-09-19T07:22:37.818 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_subcompactions: 1 2024-09-19T07:22:37.818 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T07:22:37.818 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T07:22:37.818 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T07:22:37.818 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T07:22:37.818 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T07:22:37.818 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T07:22:37.818 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T07:22:37.818 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T07:22:37.818 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_open_files: -1 2024-09-19T07:22:37.818 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T07:22:37.818 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T07:22:37.819 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T07:22:37.819 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T07:22:37.819 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_background_flushes: -1 2024-09-19T07:22:37.819 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Compression algorithms supported: 2024-09-19T07:22:37.819 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: kZSTD supported: 0 2024-09-19T07:22:37.819 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: kXpressCompression supported: 0 2024-09-19T07:22:37.819 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: kBZip2Compression supported: 0 2024-09-19T07:22:37.819 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T07:22:37.819 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: kLZ4Compression supported: 1 2024-09-19T07:22:37.819 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: kZlibCompression supported: 1 2024-09-19T07:22:37.819 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T07:22:37.819 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: kSnappyCompression supported: 1 2024-09-19T07:22:37.819 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T07:22:37.820 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T07:22:37.820 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-19T07:22:37.820 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T07:22:37.820 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T07:22:37.820 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.merge_operator: 2024-09-19T07:22:37.820 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compaction_filter: None 2024-09-19T07:22:37.820 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T07:22:37.820 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T07:22:37.820 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T07:22:37.820 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T07:22:37.820 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5615148d9300) 2024-09-19T07:22:37.820 INFO:journalctl@ceph.mon.a.smithi159.stdout: cache_index_and_filter_blocks: 1 2024-09-19T07:22:37.820 INFO:journalctl@ceph.mon.a.smithi159.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T07:22:37.821 INFO:journalctl@ceph.mon.a.smithi159.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T07:22:37.821 INFO:journalctl@ceph.mon.a.smithi159.stdout: pin_top_level_index_and_filter: 1 2024-09-19T07:22:37.821 INFO:journalctl@ceph.mon.a.smithi159.stdout: index_type: 0 2024-09-19T07:22:37.821 INFO:journalctl@ceph.mon.a.smithi159.stdout: data_block_index_type: 0 2024-09-19T07:22:37.821 INFO:journalctl@ceph.mon.a.smithi159.stdout: index_shortening: 1 2024-09-19T07:22:37.821 INFO:journalctl@ceph.mon.a.smithi159.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T07:22:37.821 INFO:journalctl@ceph.mon.a.smithi159.stdout: checksum: 4 2024-09-19T07:22:37.821 INFO:journalctl@ceph.mon.a.smithi159.stdout: no_block_cache: 0 2024-09-19T07:22:37.821 INFO:journalctl@ceph.mon.a.smithi159.stdout: block_cache: 0x5615148cd1f0 2024-09-19T07:22:37.821 INFO:journalctl@ceph.mon.a.smithi159.stdout: block_cache_name: BinnedLRUCache 2024-09-19T07:22:37.821 INFO:journalctl@ceph.mon.a.smithi159.stdout: block_cache_options: 2024-09-19T07:22:37.821 INFO:journalctl@ceph.mon.a.smithi159.stdout: capacity : 536870912 2024-09-19T07:22:37.821 INFO:journalctl@ceph.mon.a.smithi159.stdout: num_shard_bits : 4 2024-09-19T07:22:37.821 INFO:journalctl@ceph.mon.a.smithi159.stdout: strict_capacity_limit : 0 2024-09-19T07:22:37.822 INFO:journalctl@ceph.mon.a.smithi159.stdout: high_pri_pool_ratio: 0.000 2024-09-19T07:22:37.822 INFO:journalctl@ceph.mon.a.smithi159.stdout: block_cache_compressed: (nil) 2024-09-19T07:22:37.822 INFO:journalctl@ceph.mon.a.smithi159.stdout: persistent_cache: (nil) 2024-09-19T07:22:37.822 INFO:journalctl@ceph.mon.a.smithi159.stdout: block_size: 4096 2024-09-19T07:22:37.822 INFO:journalctl@ceph.mon.a.smithi159.stdout: block_size_deviation: 10 2024-09-19T07:22:37.822 INFO:journalctl@ceph.mon.a.smithi159.stdout: block_restart_interval: 16 2024-09-19T07:22:37.822 INFO:journalctl@ceph.mon.a.smithi159.stdout: index_block_restart_interval: 1 2024-09-19T07:22:37.822 INFO:journalctl@ceph.mon.a.smithi159.stdout: metadata_block_size: 4096 2024-09-19T07:22:37.822 INFO:journalctl@ceph.mon.a.smithi159.stdout: partition_filters: 0 2024-09-19T07:22:37.822 INFO:journalctl@ceph.mon.a.smithi159.stdout: use_delta_encoding: 1 2024-09-19T07:22:37.822 INFO:journalctl@ceph.mon.a.smithi159.stdout: filter_policy: bloomfilter 2024-09-19T07:22:37.822 INFO:journalctl@ceph.mon.a.smithi159.stdout: whole_key_filtering: 1 2024-09-19T07:22:37.822 INFO:journalctl@ceph.mon.a.smithi159.stdout: verify_compression: 0 2024-09-19T07:22:37.822 INFO:journalctl@ceph.mon.a.smithi159.stdout: read_amp_bytes_per_bit: 0 2024-09-19T07:22:37.823 INFO:journalctl@ceph.mon.a.smithi159.stdout: format_version: 5 2024-09-19T07:22:37.823 INFO:journalctl@ceph.mon.a.smithi159.stdout: enable_index_compression: 1 2024-09-19T07:22:37.823 INFO:journalctl@ceph.mon.a.smithi159.stdout: block_align: 0 2024-09-19T07:22:37.823 INFO:journalctl@ceph.mon.a.smithi159.stdout: max_auto_readahead_size: 262144 2024-09-19T07:22:37.823 INFO:journalctl@ceph.mon.a.smithi159.stdout: prepopulate_block_cache: 0 2024-09-19T07:22:37.823 INFO:journalctl@ceph.mon.a.smithi159.stdout: initial_auto_readahead_size: 8192 2024-09-19T07:22:37.823 INFO:journalctl@ceph.mon.a.smithi159.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T07:22:37.823 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T07:22:37.823 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T07:22:37.823 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compression: NoCompression 2024-09-19T07:22:37.823 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T07:22:37.823 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T07:22:37.823 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T07:22:37.823 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.num_levels: 7 2024-09-19T07:22:37.824 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T07:22:37.824 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T07:22:37.824 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T07:22:37.824 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T07:22:37.824 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T07:22:37.824 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T07:22:37.824 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T07:22:37.824 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T07:22:37.824 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T07:22:37.824 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T07:22:37.824 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T07:22:37.824 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T07:22:37.824 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T07:22:37.824 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T07:22:37.825 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T07:22:37.825 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T07:22:37.825 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T07:22:37.825 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T07:22:37.825 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T07:22:37.825 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T07:22:37.825 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T07:22:37.825 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T07:22:37.825 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T07:22:37.825 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T07:22:37.825 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T07:22:37.825 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T07:22:37.825 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T07:22:37.825 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T07:22:37.826 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T07:22:37.826 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T07:22:37.826 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T07:22:37.826 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T07:22:37.826 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T07:22:37.826 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T07:22:37.826 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T07:22:37.826 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T07:22:37.826 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T07:22:37.826 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T07:22:37.826 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T07:22:37.826 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T07:22:37.826 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T07:22:37.826 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T07:22:37.827 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T07:22:37.827 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T07:22:37.827 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T07:22:37.827 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T07:22:37.827 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T07:22:37.827 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T07:22:37.827 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T07:22:37.827 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T07:22:37.827 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T07:22:37.827 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T07:22:37.827 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T07:22:37.827 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T07:22:37.827 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.inplace_update_support: 0 2024-09-19T07:22:37.827 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T07:22:37.828 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T07:22:37.828 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T07:22:37.828 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T07:22:37.828 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.bloom_locality: 0 2024-09-19T07:22:37.828 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.max_successive_merges: 0 2024-09-19T07:22:37.828 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T07:22:37.828 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T07:22:37.828 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T07:22:37.828 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T07:22:37.828 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.ttl: 2592000 2024-09-19T07:22:37.828 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T07:22:37.828 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T07:22:37.828 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T07:22:37.828 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.enable_blob_files: false 2024-09-19T07:22:37.829 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.min_blob_size: 0 2024-09-19T07:22:37.829 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T07:22:37.829 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T07:22:37.829 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T07:22:37.829 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T07:22:37.829 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T07:22:37.829 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T07:22:37.829 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T07:22:37.829 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T07:22:37.829 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T07:22:37.829 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T07:22:37.829 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bd0aff7d-d155-4f5f-a2b4-279d08ba9b5a 2024-09-19T07:22:37.829 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726730557585311, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T07:22:37.829 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T07:22:37.830 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726730557586981, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100459, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98343, "index_size": 308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14312, "raw_average_key_size": 49, "raw_value_size": 90589, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726730557, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bd0aff7d-d155-4f5f-a2b4-279d08ba9b5a", "db_session_id": "YXYLXT1C4J9NHBJFECH9", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T07:22:37.830 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726730557587157, "job": 1, "event": "recovery_finished"} 2024-09-19T07:22:37.830 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T07:22:37.830 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T07:22:37.830 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5615148fce00 2024-09-19T07:22:37.830 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: DB pointer 0x5615149f8000 2024-09-19T07:22:37.830 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T07:22:37.830 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T07:22:37.830 INFO:journalctl@ceph.mon.a.smithi159.stdout: ** DB Stats ** 2024-09-19T07:22:37.830 INFO:journalctl@ceph.mon.a.smithi159.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T07:22:37.830 INFO:journalctl@ceph.mon.a.smithi159.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T07:22:37.830 INFO:journalctl@ceph.mon.a.smithi159.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T07:22:37.830 INFO:journalctl@ceph.mon.a.smithi159.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T07:22:37.831 INFO:journalctl@ceph.mon.a.smithi159.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T07:22:37.831 INFO:journalctl@ceph.mon.a.smithi159.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T07:22:37.831 INFO:journalctl@ceph.mon.a.smithi159.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T07:22:37.831 INFO:journalctl@ceph.mon.a.smithi159.stdout: 2024-09-19T07:22:37.831 INFO:journalctl@ceph.mon.a.smithi159.stdout: ** Compaction Stats [default] ** 2024-09-19T07:22:37.831 INFO:journalctl@ceph.mon.a.smithi159.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T07:22:37.831 INFO:journalctl@ceph.mon.a.smithi159.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T07:22:37.831 INFO:journalctl@ceph.mon.a.smithi159.stdout: L0 2/0 99.96 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 93.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T07:22:37.831 INFO:journalctl@ceph.mon.a.smithi159.stdout: Sum 2/0 99.96 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 93.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T07:22:37.831 INFO:journalctl@ceph.mon.a.smithi159.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 93.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T07:22:37.831 INFO:journalctl@ceph.mon.a.smithi159.stdout: 2024-09-19T07:22:37.831 INFO:journalctl@ceph.mon.a.smithi159.stdout: ** Compaction Stats [default] ** 2024-09-19T07:22:37.831 INFO:journalctl@ceph.mon.a.smithi159.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T07:22:37.832 INFO:journalctl@ceph.mon.a.smithi159.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T07:22:37.832 INFO:journalctl@ceph.mon.a.smithi159.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 93.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T07:22:37.832 INFO:journalctl@ceph.mon.a.smithi159.stdout: 2024-09-19T07:22:37.832 INFO:journalctl@ceph.mon.a.smithi159.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T07:22:37.832 INFO:journalctl@ceph.mon.a.smithi159.stdout: 2024-09-19T07:22:37.832 INFO:journalctl@ceph.mon.a.smithi159.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T07:22:37.832 INFO:journalctl@ceph.mon.a.smithi159.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T07:22:37.832 INFO:journalctl@ceph.mon.a.smithi159.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T07:22:37.832 INFO:journalctl@ceph.mon.a.smithi159.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T07:22:37.832 INFO:journalctl@ceph.mon.a.smithi159.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T07:22:37.832 INFO:journalctl@ceph.mon.a.smithi159.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T07:22:37.832 INFO:journalctl@ceph.mon.a.smithi159.stdout: Cumulative compaction: 0.00 GB write, 17.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T07:22:37.832 INFO:journalctl@ceph.mon.a.smithi159.stdout: Interval compaction: 0.00 GB write, 17.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T07:22:37.833 INFO:journalctl@ceph.mon.a.smithi159.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T07:22:37.833 INFO:journalctl@ceph.mon.a.smithi159.stdout: Block cache BinnedLRUCache@0x5615148cd1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-19T07:22:37.833 INFO:journalctl@ceph.mon.a.smithi159.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-19T07:22:37.833 INFO:journalctl@ceph.mon.a.smithi159.stdout: 2024-09-19T07:22:37.833 INFO:journalctl@ceph.mon.a.smithi159.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T07:22:37.833 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: starting mon.a rank 0 at public addrs [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] at bind addrs [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:22:37.833 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: mon.a@-1(???) e1 preinit fsid c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:22:37.833 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: mon.a@-1(???).mds e1 new map 2024-09-19T07:22:37.833 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: mon.a@-1(???).mds e1 print_map 2024-09-19T07:22:37.833 INFO:journalctl@ceph.mon.a.smithi159.stdout: e1 2024-09-19T07:22:37.833 INFO:journalctl@ceph.mon.a.smithi159.stdout: btime 2024-09-19T07:22:32:059587+0000 2024-09-19T07:22:37.833 INFO:journalctl@ceph.mon.a.smithi159.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T07:22:37.833 INFO:journalctl@ceph.mon.a.smithi159.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T07:22:37.834 INFO:journalctl@ceph.mon.a.smithi159.stdout: legacy client fscid: -1 2024-09-19T07:22:37.834 INFO:journalctl@ceph.mon.a.smithi159.stdout: 2024-09-19T07:22:37.834 INFO:journalctl@ceph.mon.a.smithi159.stdout: No filesystems configured 2024-09-19T07:22:37.834 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T07:22:37.834 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T07:22:37.834 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T07:22:37.834 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T07:22:37.834 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T07:22:37.834 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T07:22:37.834 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: monmap epoch 1 2024-09-19T07:22:37.834 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: fsid c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:22:37.834 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: last_changed 2024-09-19T07:22:28.312056+0000 2024-09-19T07:22:37.834 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: created 2024-09-19T07:22:28.312056+0000 2024-09-19T07:22:37.834 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: min_mon_release 19 (squid) 2024-09-19T07:22:37.834 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: election_strategy: 1 2024-09-19T07:22:37.835 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: 0: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.a 2024-09-19T07:22:37.835 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: fsmap 2024-09-19T07:22:37.835 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T07:22:37.835 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 ceph-mon[30472]: mgrmap e1: no daemons active 2024-09-19T07:22:37.835 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 bash[30453]: 37afdc5581e3e9dde80d4424cafe97f73df3462a265d8296a8ddb3d59549ac06 2024-09-19T07:22:37.835 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:37 smithi159 systemd[1]: Started Ceph mon.a for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:22:38.961 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.240+0000 7f224e800640 1 Processor -- start 2024-09-19T07:22:38.961 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.240+0000 7f224e800640 1 -- start start 2024-09-19T07:22:38.961 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.241+0000 7f224e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f224807dde0 0x7f224807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:38.962 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.241+0000 7f224e800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f224807c820 con 0x7f224807dde0 2024-09-19T07:22:38.962 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.241+0000 7f2247400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f224807dde0 0x7f224807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:38.962 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.241+0000 7f2247400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f224807dde0 0x7f224807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42466/0 (socket says 172.21.15.159:42466) 2024-09-19T07:22:38.962 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.241+0000 7f2247400640 1 -- 172.21.15.159:0/918674559 learned_addr learned my addr 172.21.15.159:0/918674559 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:38.962 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.242+0000 7f2247400640 1 -- 172.21.15.159:0/918674559 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224807c9a0 con 0x7f224807dde0 2024-09-19T07:22:38.962 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.245+0000 7f2247400640 1 --2- 172.21.15.159:0/918674559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f224807dde0 0x7f224807c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2238009820 tx=0x7f223802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfffd48f89bc45d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:38.962 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.245+0000 7f2246000640 1 -- 172.21.15.159:0/918674559 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2238037070 con 0x7f224807dde0 2024-09-19T07:22:38.962 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.245+0000 7f2246000640 1 -- 172.21.15.159:0/918674559 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2238032900 con 0x7f224807dde0 2024-09-19T07:22:38.962 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.245+0000 7f2246000640 1 -- 172.21.15.159:0/918674559 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2238030850 con 0x7f224807dde0 2024-09-19T07:22:38.962 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.245+0000 7f224e800640 1 -- 172.21.15.159:0/918674559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f224807dde0 msgr2=0x7f224807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:38.962 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.245+0000 7f224e800640 1 --2- 172.21.15.159:0/918674559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f224807dde0 0x7f224807c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2238009820 tx=0x7f223802a270 comp rx=0 tx=0).stop 2024-09-19T07:22:38.963 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.245+0000 7f224e800640 1 -- 172.21.15.159:0/918674559 shutdown_connections 2024-09-19T07:22:38.963 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.245+0000 7f224e800640 1 --2- 172.21.15.159:0/918674559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f224807dde0 0x7f224807c2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:38.963 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.245+0000 7f224e800640 1 -- 172.21.15.159:0/918674559 >> 172.21.15.159:0/918674559 conn(0x7f2248102e50 msgr2=0x7f2248105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:38.963 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.245+0000 7f224e800640 1 -- 172.21.15.159:0/918674559 shutdown_connections 2024-09-19T07:22:38.963 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.245+0000 7f224e800640 1 -- 172.21.15.159:0/918674559 wait complete. 2024-09-19T07:22:38.963 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.246+0000 7f224e800640 1 Processor -- start 2024-09-19T07:22:38.963 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.246+0000 7f224e800640 1 -- start start 2024-09-19T07:22:38.963 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.247+0000 7f224e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f224807dde0 0x7f224819d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:38.963 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.247+0000 7f224e800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f224810dc70 con 0x7f224807dde0 2024-09-19T07:22:38.963 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.247+0000 7f2247400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f224807dde0 0x7f224819d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:38.963 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.247+0000 7f2247400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f224807dde0 0x7f224819d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42472/0 (socket says 172.21.15.159:42472) 2024-09-19T07:22:38.963 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.247+0000 7f2247400640 1 -- 172.21.15.159:0/3413472161 learned_addr learned my addr 172.21.15.159:0/3413472161 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:38.963 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.247+0000 7f2247400640 1 -- 172.21.15.159:0/3413472161 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224819dd60 con 0x7f224807dde0 2024-09-19T07:22:38.964 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.248+0000 7f2247400640 1 --2- 172.21.15.159:0/3413472161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f224807dde0 0x7f224819d820 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2238031000 tx=0x7f22380326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:38.964 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.248+0000 7f223fe00640 1 -- 172.21.15.159:0/3413472161 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2238042070 con 0x7f224807dde0 2024-09-19T07:22:38.964 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.248+0000 7f223fe00640 1 -- 172.21.15.159:0/3413472161 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2238032be0 con 0x7f224807dde0 2024-09-19T07:22:38.964 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.248+0000 7f223fe00640 1 -- 172.21.15.159:0/3413472161 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2238037040 con 0x7f224807dde0 2024-09-19T07:22:38.964 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.248+0000 7f224e800640 1 -- 172.21.15.159:0/3413472161 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224819dff0 con 0x7f224807dde0 2024-09-19T07:22:38.964 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.248+0000 7f224e800640 1 -- 172.21.15.159:0/3413472161 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22481a0ce0 con 0x7f224807dde0 2024-09-19T07:22:38.964 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.249+0000 7f223fe00640 1 -- 172.21.15.159:0/3413472161 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2238045070 con 0x7f224807dde0 2024-09-19T07:22:38.964 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.249+0000 7f223fe00640 1 -- 172.21.15.159:0/3413472161 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2238047830 con 0x7f224807dde0 2024-09-19T07:22:38.964 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.249+0000 7f224e800640 1 -- 172.21.15.159:0/3413472161 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f224807c360 con 0x7f224807dde0 2024-09-19T07:22:38.964 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.252+0000 7f223fe00640 1 -- 172.21.15.159:0/3413472161 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f223804c030 con 0x7f224807dde0 2024-09-19T07:22:38.964 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.312+0000 7f224e800640 1 -- 172.21.15.159:0/3413472161 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f22481a1000 con 0x7f224807dde0 2024-09-19T07:22:38.964 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.315+0000 7f223fe00640 1 -- 172.21.15.159:0/3413472161 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f2238032d90 con 0x7f224807dde0 2024-09-19T07:22:38.964 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.315+0000 7f224e800640 1 -- 172.21.15.159:0/3413472161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f224807dde0 msgr2=0x7f224819d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:38.964 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.315+0000 7f224e800640 1 --2- 172.21.15.159:0/3413472161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f224807dde0 0x7f224819d820 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2238031000 tx=0x7f22380326e0 comp rx=0 tx=0).stop 2024-09-19T07:22:38.965 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.315+0000 7f224e800640 1 -- 172.21.15.159:0/3413472161 shutdown_connections 2024-09-19T07:22:38.965 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.315+0000 7f224e800640 1 --2- 172.21.15.159:0/3413472161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f224807dde0 0x7f224819d820 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:38.965 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.315+0000 7f224e800640 1 -- 172.21.15.159:0/3413472161 >> 172.21.15.159:0/3413472161 conn(0x7f2248102e50 msgr2=0x7f2248103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:38.965 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.316+0000 7f224e800640 1 -- 172.21.15.159:0/3413472161 shutdown_connections 2024-09-19T07:22:38.965 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:38.316+0000 7f224e800640 1 -- 172.21.15.159:0/3413472161 wait complete. 2024-09-19T07:22:38.965 INFO:teuthology.orchestra.run.smithi159.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T07:22:38.984 INFO:teuthology.orchestra.run.smithi159.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:22:38.984 INFO:teuthology.orchestra.run.smithi159.stdout:Creating mgr... 2024-09-19T07:22:38.985 INFO:teuthology.orchestra.run.smithi159.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T07:22:38.985 INFO:teuthology.orchestra.run.smithi159.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T07:22:39.333 INFO:teuthology.orchestra.run.smithi159.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c9d18048-7657-11ef-bceb-c7b262605968@mgr.x 2024-09-19T07:22:39.333 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c9d18048-7657-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-c9d18048-7657-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-19T07:22:39.571 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c9d18048-7657-11ef-bceb-c7b262605968.target.wants/ceph-c9d18048-7657-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-c9d18048-7657-11ef-bceb-c7b262605968@.service. 2024-09-19T07:22:39.605 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:39 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3413472161' entity='client.admin' 2024-09-19T07:22:39.887 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:39 smithi159 systemd[1]: Starting Ceph mgr.x for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:22:40.206 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:39 smithi159 podman[30681]: 2024-09-19 07:22:39.821322609 +0000 UTC m=+0.027731178 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:22:40.206 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:39 smithi159 podman[30681]: 2024-09-19 07:22:39.945656736 +0000 UTC m=+0.152065313 container create 69b92e6f3945fd74c1b2acc742cf7d78781715e0fb36f654e6e56351f3fac875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD) 2024-09-19T07:22:40.206 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:40 smithi159 podman[30681]: 2024-09-19 07:22:40.123274249 +0000 UTC m=+0.329682819 container init 69b92e6f3945fd74c1b2acc742cf7d78781715e0fb36f654e6e56351f3fac875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T07:22:40.206 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:40 smithi159 podman[30681]: 2024-09-19 07:22:40.129493006 +0000 UTC m=+0.335901577 container start 69b92e6f3945fd74c1b2acc742cf7d78781715e0fb36f654e6e56351f3fac875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-19T07:22:40.206 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:40 smithi159 bash[30681]: 69b92e6f3945fd74c1b2acc742cf7d78781715e0fb36f654e6e56351f3fac875 2024-09-19T07:22:40.234 INFO:teuthology.orchestra.run.smithi159.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T07:22:40.234 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stdout disabled 2024-09-19T07:22:40.249 INFO:teuthology.orchestra.run.smithi159.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T07:22:40.249 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stdout inactive 2024-09-19T07:22:40.249 INFO:teuthology.orchestra.run.smithi159.stdout:firewalld.service is not enabled 2024-09-19T07:22:40.250 INFO:teuthology.orchestra.run.smithi159.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T07:22:40.268 INFO:teuthology.orchestra.run.smithi159.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T07:22:40.268 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stdout disabled 2024-09-19T07:22:40.284 INFO:teuthology.orchestra.run.smithi159.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T07:22:40.284 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stdout inactive 2024-09-19T07:22:40.284 INFO:teuthology.orchestra.run.smithi159.stdout:firewalld.service is not enabled 2024-09-19T07:22:40.284 INFO:teuthology.orchestra.run.smithi159.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-19T07:22:40.284 INFO:teuthology.orchestra.run.smithi159.stdout:Waiting for mgr to start... 2024-09-19T07:22:40.284 INFO:teuthology.orchestra.run.smithi159.stdout:Waiting for mgr... 2024-09-19T07:22:40.541 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:40 smithi159 systemd[1]: Started Ceph mgr.x for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:22:40.542 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:40 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:40.314+0000 7f4ee9e36040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T07:22:40.542 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:40 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:40.393+0000 7f4ee9e36040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T07:22:40.800 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:40 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:40.590+0000 7f4ee9e36040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T07:22:40.801 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:40 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:40.671+0000 7f4ee9e36040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T07:22:41.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:40 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/503381449' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T07:22:41.137 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:40 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:40.856+0000 7f4ee9e36040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:22:41.137 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:41 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:41.032+0000 7f4ee9e36040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T07:22:41.636 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:41 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:41.420+0000 7f4ee9e36040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T07:22:41.637 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:41 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:41.492+0000 7f4ee9e36040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T07:22:41.643 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout 2024-09-19T07:22:41.645 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout { 2024-09-19T07:22:41.645 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "fsid": "c9d18048-7657-11ef-bceb-c7b262605968", 2024-09-19T07:22:41.646 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T07:22:41.646 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T07:22:41.646 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T07:22:41.646 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T07:22:41.646 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:22:41.646 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T07:22:41.646 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T07:22:41.646 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout 0 2024-09-19T07:22:41.646 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout ], 2024-09-19T07:22:41.646 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T07:22:41.646 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T07:22:41.646 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout ], 2024-09-19T07:22:41.646 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-19T07:22:41.646 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T07:22:41.647 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:22:41.647 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T07:22:41.647 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T07:22:41.647 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:22:41.647 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T07:22:41.647 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:22:41.647 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T07:22:41.647 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T07:22:41.647 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T07:22:41.654 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T07:22:41.655 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T07:22:41.655 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T07:22:41.655 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:22:41.655 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T07:22:41.655 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T07:22:41.655 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T07:22:41.655 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T07:22:41.655 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T07:22:41.655 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T07:22:41.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T07:22:41.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T07:22:41.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T07:22:41.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:22:41.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T07:22:41.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:22:41.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T07:22:32:059587+0000", 2024-09-19T07:22:41.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T07:22:41.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T07:22:41.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:22:41.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T07:22:41.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T07:22:41.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T07:22:41.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T07:22:41.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T07:22:41.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T07:22:41.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T07:22:41.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout ], 2024-09-19T07:22:41.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T07:22:41.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:22:41.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T07:22:41.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:22:41.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T07:22:32.060353+0000", 2024-09-19T07:22:41.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T07:22:41.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:22:41.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T07:22:41.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout } 2024-09-19T07:22:41.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.818+0000 7fa3df200640 1 Processor -- start 2024-09-19T07:22:41.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.818+0000 7fa3df200640 1 -- start start 2024-09-19T07:22:41.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.819+0000 7fa3df200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e0107440 0x7fa3e0107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:41.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.819+0000 7fa3df200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e0107e10 con 0x7fa3e0107440 2024-09-19T07:22:41.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.819+0000 7fa3dde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e0107440 0x7fa3e0107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:41.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.819+0000 7fa3dde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e0107440 0x7fa3e0107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42504/0 (socket says 172.21.15.159:42504) 2024-09-19T07:22:41.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.819+0000 7fa3dde00640 1 -- 172.21.15.159:0/2237629328 learned_addr learned my addr 172.21.15.159:0/2237629328 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:41.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.820+0000 7fa3dde00640 1 -- 172.21.15.159:0/2237629328 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e0108640 con 0x7fa3e0107440 2024-09-19T07:22:41.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.821+0000 7fa3dde00640 1 --2- 172.21.15.159:0/2237629328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e0107440 0x7fa3e0107840 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009820 tx=0x7fa3d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b60faeaea701cd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:41.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.821+0000 7fa3dca00640 1 -- 172.21.15.159:0/2237629328 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3d0037070 con 0x7fa3e0107440 2024-09-19T07:22:41.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.821+0000 7fa3dca00640 1 -- 172.21.15.159:0/2237629328 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa3d0032900 con 0x7fa3e0107440 2024-09-19T07:22:41.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.821+0000 7fa3dca00640 1 -- 172.21.15.159:0/2237629328 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3d0030850 con 0x7fa3e0107440 2024-09-19T07:22:41.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.822+0000 7fa3df200640 1 -- 172.21.15.159:0/2237629328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e0107440 msgr2=0x7fa3e0107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:41.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.822+0000 7fa3df200640 1 --2- 172.21.15.159:0/2237629328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e0107440 0x7fa3e0107840 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009820 tx=0x7fa3d002a270 comp rx=0 tx=0).stop 2024-09-19T07:22:41.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.822+0000 7fa3df200640 1 -- 172.21.15.159:0/2237629328 shutdown_connections 2024-09-19T07:22:41.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.822+0000 7fa3df200640 1 --2- 172.21.15.159:0/2237629328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e0107440 0x7fa3e0107840 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:41.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.822+0000 7fa3df200640 1 -- 172.21.15.159:0/2237629328 >> 172.21.15.159:0/2237629328 conn(0x7fa3e0102c10 msgr2=0x7fa3e0105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:41.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.822+0000 7fa3df200640 1 -- 172.21.15.159:0/2237629328 shutdown_connections 2024-09-19T07:22:41.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.822+0000 7fa3df200640 1 -- 172.21.15.159:0/2237629328 wait complete. 2024-09-19T07:22:41.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.823+0000 7fa3df200640 1 Processor -- start 2024-09-19T07:22:41.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.823+0000 7fa3df200640 1 -- start start 2024-09-19T07:22:41.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.823+0000 7fa3df200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e0107440 0x7fa3e019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:41.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.823+0000 7fa3df200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e010b9f0 con 0x7fa3e0107440 2024-09-19T07:22:41.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.824+0000 7fa3dde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e0107440 0x7fa3e019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:41.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.824+0000 7fa3dde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e0107440 0x7fa3e019d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42506/0 (socket says 172.21.15.159:42506) 2024-09-19T07:22:41.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.824+0000 7fa3dde00640 1 -- 172.21.15.159:0/503381449 learned_addr learned my addr 172.21.15.159:0/503381449 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:41.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.824+0000 7fa3dde00640 1 -- 172.21.15.159:0/503381449 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e019dc50 con 0x7fa3e0107440 2024-09-19T07:22:41.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.825+0000 7fa3dde00640 1 --2- 172.21.15.159:0/503381449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e0107440 0x7fa3e019d710 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00097f0 tx=0x7fa3d0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:41.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.825+0000 7fa3d6a00640 1 -- 172.21.15.159:0/503381449 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3d0042070 con 0x7fa3e0107440 2024-09-19T07:22:41.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.825+0000 7fa3d6a00640 1 -- 172.21.15.159:0/503381449 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa3d0032ca0 con 0x7fa3e0107440 2024-09-19T07:22:41.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.825+0000 7fa3df200640 1 -- 172.21.15.159:0/503381449 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e019dee0 con 0x7fa3e0107440 2024-09-19T07:22:41.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.825+0000 7fa3d6a00640 1 -- 172.21.15.159:0/503381449 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3d0037040 con 0x7fa3e0107440 2024-09-19T07:22:41.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.825+0000 7fa3df200640 1 -- 172.21.15.159:0/503381449 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e01a0bd0 con 0x7fa3e0107440 2024-09-19T07:22:41.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.825+0000 7fa3d6a00640 1 -- 172.21.15.159:0/503381449 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa3d0045070 con 0x7fa3e0107440 2024-09-19T07:22:41.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.825+0000 7fa3d6a00640 1 -- 172.21.15.159:0/503381449 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa3d0047a30 con 0x7fa3e0107440 2024-09-19T07:22:41.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.826+0000 7fa3df200640 1 -- 172.21.15.159:0/503381449 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e010bb70 con 0x7fa3e0107440 2024-09-19T07:22:41.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.829+0000 7fa3d6a00640 1 -- 172.21.15.159:0/503381449 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa3d003da60 con 0x7fa3e0107440 2024-09-19T07:22:41.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.893+0000 7fa3df200640 1 -- 172.21.15.159:0/503381449 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa3e01a10b0 con 0x7fa3e0107440 2024-09-19T07:22:41.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.894+0000 7fa3d6a00640 1 -- 172.21.15.159:0/503381449 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fa3d003d430 con 0x7fa3e0107440 2024-09-19T07:22:41.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.895+0000 7fa3df200640 1 -- 172.21.15.159:0/503381449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e0107440 msgr2=0x7fa3e019d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:41.661 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.895+0000 7fa3df200640 1 --2- 172.21.15.159:0/503381449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e0107440 0x7fa3e019d710 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00097f0 tx=0x7fa3d0030ec0 comp rx=0 tx=0).stop 2024-09-19T07:22:41.661 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.895+0000 7fa3df200640 1 -- 172.21.15.159:0/503381449 shutdown_connections 2024-09-19T07:22:41.661 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.895+0000 7fa3df200640 1 --2- 172.21.15.159:0/503381449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e0107440 0x7fa3e019d710 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:41.661 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.895+0000 7fa3df200640 1 -- 172.21.15.159:0/503381449 >> 172.21.15.159:0/503381449 conn(0x7fa3e0102c10 msgr2=0x7fa3e01037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:41.661 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.895+0000 7fa3df200640 1 -- 172.21.15.159:0/503381449 shutdown_connections 2024-09-19T07:22:41.661 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:40.895+0000 7fa3df200640 1 -- 172.21.15.159:0/503381449 wait complete. 2024-09-19T07:22:41.661 INFO:teuthology.orchestra.run.smithi159.stdout:mgr not available, waiting (1/15)... 2024-09-19T07:22:42.136 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:41 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:41.710+0000 7f4ee9e36040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T07:22:43.137 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:42 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:42.820+0000 7f4ee9e36040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T07:22:43.137 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:42 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:42.900+0000 7f4ee9e36040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T07:22:44.136 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:43 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:43.862+0000 7f4ee9e36040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T07:22:44.137 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:44 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:44.003+0000 7f4ee9e36040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T07:22:44.426 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:44 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:44.174+0000 7f4ee9e36040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:22:44.427 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:44 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:44.249+0000 7f4ee9e36040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T07:22:44.427 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:44 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:44.426+0000 7f4ee9e36040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T07:22:44.886 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:44 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:44.510+0000 7f4ee9e36040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T07:22:44.887 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:44 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:44.585+0000 7f4ee9e36040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T07:22:45.180 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:44 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:44.767+0000 7f4ee9e36040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T07:22:45.181 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:44 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:44.845+0000 7f4ee9e36040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T07:22:45.181 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:44 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:44.933+0000 7f4ee9e36040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T07:22:45.181 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:45 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:45.037+0000 7f4ee9e36040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T07:22:45.181 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:45 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:45.110+0000 7f4ee9e36040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T07:22:45.181 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:45 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:45.180+0000 7f4ee9e36040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T07:22:46.637 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:46 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:46.159+0000 7f4ee9e36040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T07:22:46.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:46 smithi159 ceph-mon[30472]: Activating manager daemon x 2024-09-19T07:22:46.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:46 smithi159 ceph-mon[30472]: mgrmap e2: x(active, starting, since 0.00441821s) 2024-09-19T07:22:46.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:46 smithi159 ceph-mon[30472]: from='mgr.14100 172.21.15.159:0/2735578731' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T07:22:46.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:46 smithi159 ceph-mon[30472]: from='mgr.14100 172.21.15.159:0/2735578731' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T07:22:46.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:46 smithi159 ceph-mon[30472]: from='mgr.14100 172.21.15.159:0/2735578731' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T07:22:46.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:46 smithi159 ceph-mon[30472]: from='mgr.14100 172.21.15.159:0/2735578731' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:22:46.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:46 smithi159 ceph-mon[30472]: from='mgr.14100 172.21.15.159:0/2735578731' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T07:22:46.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:46 smithi159 ceph-mon[30472]: Manager daemon x is now available 2024-09-19T07:22:46.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:46 smithi159 ceph-mon[30472]: from='mgr.14100 172.21.15.159:0/2735578731' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T07:22:46.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:46 smithi159 ceph-mon[30472]: from='mgr.14100 172.21.15.159:0/2735578731' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T07:22:46.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:46 smithi159 ceph-mon[30472]: from='mgr.14100 172.21.15.159:0/2735578731' entity='mgr.x' 2024-09-19T07:22:46.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:46 smithi159 ceph-mon[30472]: from='mgr.14100 172.21.15.159:0/2735578731' entity='mgr.x' 2024-09-19T07:22:47.490 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:47 smithi159 ceph-mon[30472]: from='mgr.14100 172.21.15.159:0/2735578731' entity='mgr.x' 2024-09-19T07:22:47.490 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:47 smithi159 ceph-mon[30472]: mgrmap e3: x(active, since 1.01019s) 2024-09-19T07:22:49.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:49 smithi159 ceph-mon[30472]: mgrmap e4: x(active, since 2s) 2024-09-19T07:22:54.723 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:54 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2089865104' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T07:22:55.010 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout 2024-09-19T07:22:55.013 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout { 2024-09-19T07:22:55.013 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "fsid": "c9d18048-7657-11ef-bceb-c7b262605968", 2024-09-19T07:22:55.013 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T07:22:55.013 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T07:22:55.013 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T07:22:55.013 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T07:22:55.013 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:22:55.013 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T07:22:55.013 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T07:22:55.014 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout 0 2024-09-19T07:22:55.014 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout ], 2024-09-19T07:22:55.014 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T07:22:55.014 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T07:22:55.014 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout ], 2024-09-19T07:22:55.014 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2024-09-19T07:22:55.014 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T07:22:55.014 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:22:55.014 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T07:22:55.014 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T07:22:55.014 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:22:55.014 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T07:22:55.014 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:22:55.014 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T07:22:55.015 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T07:22:55.015 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T07:22:55.027 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T07:22:55.027 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T07:22:55.027 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T07:22:55.027 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:22:55.027 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T07:22:55.027 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T07:22:55.027 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T07:22:55.027 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T07:22:55.028 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T07:22:55.028 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T07:22:55.028 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T07:22:55.028 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T07:22:55.028 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T07:22:55.028 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:22:55.028 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T07:22:55.028 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:22:55.028 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T07:22:32:059587+0000", 2024-09-19T07:22:55.028 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T07:22:55.028 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T07:22:55.028 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:22:55.028 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T07:22:55.028 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T07:22:55.029 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T07:22:55.029 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T07:22:55.029 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T07:22:55.029 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T07:22:55.029 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T07:22:55.029 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout ], 2024-09-19T07:22:55.029 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T07:22:55.029 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:22:55.029 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T07:22:55.029 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:22:55.029 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T07:22:32.060353+0000", 2024-09-19T07:22:55.029 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T07:22:55.029 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:22:55.030 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T07:22:55.030 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout } 2024-09-19T07:22:55.030 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.137+0000 7f071ee00640 1 Processor -- start 2024-09-19T07:22:55.030 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.137+0000 7f071ee00640 1 -- start start 2024-09-19T07:22:55.030 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.138+0000 7f071ee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f071807dd70 0x7f071807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:55.030 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.138+0000 7f071ee00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f071807e170 con 0x7f071807dd70 2024-09-19T07:22:55.030 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.138+0000 7f0717e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f071807dd70 0x7f071807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:55.030 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.138+0000 7f0717e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f071807dd70 0x7f071807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52706/0 (socket says 172.21.15.159:52706) 2024-09-19T07:22:55.030 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.138+0000 7f0717e00640 1 -- 172.21.15.159:0/2146113620 learned_addr learned my addr 172.21.15.159:0/2146113620 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:55.030 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.139+0000 7f0717e00640 1 -- 172.21.15.159:0/2146113620 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071807c820 con 0x7f071807dd70 2024-09-19T07:22:55.030 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.140+0000 7f0717e00640 1 --2- 172.21.15.159:0/2146113620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f071807dd70 0x7f071807c2e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0708009820 tx=0x7f070802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c338de0e896bd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:55.030 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.140+0000 7f0716a00640 1 -- 172.21.15.159:0/2146113620 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0708037070 con 0x7f071807dd70 2024-09-19T07:22:55.030 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.140+0000 7f0716a00640 1 -- 172.21.15.159:0/2146113620 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0708032900 con 0x7f071807dd70 2024-09-19T07:22:55.030 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.140+0000 7f0716a00640 1 -- 172.21.15.159:0/2146113620 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0708030850 con 0x7f071807dd70 2024-09-19T07:22:55.031 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.141+0000 7f071ee00640 1 -- 172.21.15.159:0/2146113620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f071807dd70 msgr2=0x7f071807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:55.031 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.141+0000 7f071ee00640 1 --2- 172.21.15.159:0/2146113620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f071807dd70 0x7f071807c2e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0708009820 tx=0x7f070802a270 comp rx=0 tx=0).stop 2024-09-19T07:22:55.031 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.141+0000 7f071ee00640 1 -- 172.21.15.159:0/2146113620 shutdown_connections 2024-09-19T07:22:55.031 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.141+0000 7f071ee00640 1 --2- 172.21.15.159:0/2146113620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f071807dd70 0x7f071807c2e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:55.031 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.141+0000 7f071ee00640 1 -- 172.21.15.159:0/2146113620 >> 172.21.15.159:0/2146113620 conn(0x7f0718102eb0 msgr2=0x7f07181052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:55.031 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.141+0000 7f071ee00640 1 -- 172.21.15.159:0/2146113620 shutdown_connections 2024-09-19T07:22:55.031 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.141+0000 7f071ee00640 1 -- 172.21.15.159:0/2146113620 wait complete. 2024-09-19T07:22:55.031 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.142+0000 7f071ee00640 1 Processor -- start 2024-09-19T07:22:55.031 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.142+0000 7f071ee00640 1 -- start start 2024-09-19T07:22:55.031 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.142+0000 7f071ee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f071807dd70 0x7f071819d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:55.031 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.142+0000 7f071ee00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f071810dbc0 con 0x7f071807dd70 2024-09-19T07:22:55.031 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.143+0000 7f0717e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f071807dd70 0x7f071819d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:55.032 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.143+0000 7f0717e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f071807dd70 0x7f071819d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52720/0 (socket says 172.21.15.159:52720) 2024-09-19T07:22:55.032 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.143+0000 7f0717e00640 1 -- 172.21.15.159:0/2089865104 learned_addr learned my addr 172.21.15.159:0/2089865104 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:55.032 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.143+0000 7f0717e00640 1 -- 172.21.15.159:0/2089865104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071819ddb0 con 0x7f071807dd70 2024-09-19T07:22:55.032 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.143+0000 7f0717e00640 1 --2- 172.21.15.159:0/2089865104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f071807dd70 0x7f071819d870 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0708031000 tx=0x7f07080326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:55.032 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.143+0000 7f0714c00640 1 -- 172.21.15.159:0/2089865104 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0708042070 con 0x7f071807dd70 2024-09-19T07:22:55.032 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.143+0000 7f0714c00640 1 -- 172.21.15.159:0/2089865104 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0708032be0 con 0x7f071807dd70 2024-09-19T07:22:55.032 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.143+0000 7f0714c00640 1 -- 172.21.15.159:0/2089865104 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0708037040 con 0x7f071807dd70 2024-09-19T07:22:55.032 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.144+0000 7f071ee00640 1 -- 172.21.15.159:0/2089865104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071819e040 con 0x7f071807dd70 2024-09-19T07:22:55.032 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.144+0000 7f071ee00640 1 -- 172.21.15.159:0/2089865104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07181a0d30 con 0x7f071807dd70 2024-09-19T07:22:55.032 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.145+0000 7f0714c00640 1 -- 172.21.15.159:0/2089865104 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f0708045070 con 0x7f071807dd70 2024-09-19T07:22:55.032 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.145+0000 7f071ee00640 1 -- 172.21.15.159:0/2089865104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f071807c360 con 0x7f071807dd70 2024-09-19T07:22:55.033 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.145+0000 7f0714c00640 1 --2- 172.21.15.159:0/2089865104 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f06fc039160 0x7f06fc03b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:55.033 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.145+0000 7f0714c00640 1 -- 172.21.15.159:0/2089865104 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f070806c230 con 0x7f071807dd70 2024-09-19T07:22:55.033 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.145+0000 7f0717400640 1 --2- 172.21.15.159:0/2089865104 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f06fc039160 0x7f06fc03b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:55.033 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.146+0000 7f0717400640 1 --2- 172.21.15.159:0/2089865104 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f06fc039160 0x7f06fc03b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f07040098c0 tx=0x7f0704006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:55.033 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.151+0000 7f0714c00640 1 -- 172.21.15.159:0/2089865104 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0708032d90 con 0x7f071807dd70 2024-09-19T07:22:55.033 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.391+0000 7f071ee00640 1 -- 172.21.15.159:0/2089865104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f07181a13b0 con 0x7f071807dd70 2024-09-19T07:22:55.033 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.391+0000 7f0714c00640 1 -- 172.21.15.159:0/2089865104 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f070806bc50 con 0x7f071807dd70 2024-09-19T07:22:55.033 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.393+0000 7f071ee00640 1 -- 172.21.15.159:0/2089865104 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f06fc039160 msgr2=0x7f06fc03b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:55.033 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.393+0000 7f071ee00640 1 --2- 172.21.15.159:0/2089865104 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f06fc039160 0x7f06fc03b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f07040098c0 tx=0x7f0704006db0 comp rx=0 tx=0).stop 2024-09-19T07:22:55.034 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.393+0000 7f071ee00640 1 -- 172.21.15.159:0/2089865104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f071807dd70 msgr2=0x7f071819d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:55.034 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.393+0000 7f071ee00640 1 --2- 172.21.15.159:0/2089865104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f071807dd70 0x7f071819d870 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0708031000 tx=0x7f07080326e0 comp rx=0 tx=0).stop 2024-09-19T07:22:55.034 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.393+0000 7f071ee00640 1 -- 172.21.15.159:0/2089865104 shutdown_connections 2024-09-19T07:22:55.034 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.393+0000 7f071ee00640 1 --2- 172.21.15.159:0/2089865104 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f06fc039160 0x7f06fc03b620 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:55.034 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.393+0000 7f071ee00640 1 --2- 172.21.15.159:0/2089865104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f071807dd70 0x7f071819d870 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:55.034 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.393+0000 7f071ee00640 1 -- 172.21.15.159:0/2089865104 >> 172.21.15.159:0/2089865104 conn(0x7f0718102eb0 msgr2=0x7f0718103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:55.034 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.394+0000 7f071ee00640 1 -- 172.21.15.159:0/2089865104 shutdown_connections 2024-09-19T07:22:55.034 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:54.394+0000 7f071ee00640 1 -- 172.21.15.159:0/2089865104 wait complete. 2024-09-19T07:22:55.034 INFO:teuthology.orchestra.run.smithi159.stdout:mgr is available 2024-09-19T07:22:56.042 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:55 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/4255081318' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T07:22:56.384 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout 2024-09-19T07:22:56.385 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T07:22:56.385 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout fsid = c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:22:56.385 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.159:3300,v1:172.21.15.159:6789] 2024-09-19T07:22:56.385 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T07:22:56.385 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T07:22:56.385 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T07:22:56.385 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T07:22:56.385 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout 2024-09-19T07:22:56.385 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T07:22:56.386 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T07:22:56.386 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout 2024-09-19T07:22:56.386 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T07:22:56.386 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T07:22:56.386 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T07:22:56.386 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T07:22:56.386 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T07:22:56.386 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T07:22:56.386 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.505+0000 7f399e000640 1 Processor -- start 2024-09-19T07:22:56.386 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.505+0000 7f399e000640 1 -- start start 2024-09-19T07:22:56.386 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.505+0000 7f399e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3998107440 0x7f3998107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:56.386 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.505+0000 7f399e000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3998107e10 con 0x7f3998107440 2024-09-19T07:22:56.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.505+0000 7f399cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3998107440 0x7f3998107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:56.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.506+0000 7f399cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3998107440 0x7f3998107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52736/0 (socket says 172.21.15.159:52736) 2024-09-19T07:22:56.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.506+0000 7f399cc00640 1 -- 172.21.15.159:0/2318742583 learned_addr learned my addr 172.21.15.159:0/2318742583 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:56.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.506+0000 7f399cc00640 1 -- 172.21.15.159:0/2318742583 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3998108640 con 0x7f3998107440 2024-09-19T07:22:56.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.507+0000 7f399cc00640 1 --2- 172.21.15.159:0/2318742583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3998107440 0x7f3998107840 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3988009820 tx=0x7f398802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480738aa0416af9f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:56.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.507+0000 7f3997e00640 1 -- 172.21.15.159:0/2318742583 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3988037070 con 0x7f3998107440 2024-09-19T07:22:56.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.507+0000 7f3997e00640 1 -- 172.21.15.159:0/2318742583 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3988032900 con 0x7f3998107440 2024-09-19T07:22:56.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.507+0000 7f3997e00640 1 -- 172.21.15.159:0/2318742583 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3988030850 con 0x7f3998107440 2024-09-19T07:22:56.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.508+0000 7f399e000640 1 -- 172.21.15.159:0/2318742583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3998107440 msgr2=0x7f3998107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:56.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.508+0000 7f399e000640 1 --2- 172.21.15.159:0/2318742583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3998107440 0x7f3998107840 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3988009820 tx=0x7f398802a270 comp rx=0 tx=0).stop 2024-09-19T07:22:56.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.508+0000 7f399e000640 1 -- 172.21.15.159:0/2318742583 shutdown_connections 2024-09-19T07:22:56.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.508+0000 7f399e000640 1 --2- 172.21.15.159:0/2318742583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3998107440 0x7f3998107840 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:56.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.508+0000 7f399e000640 1 -- 172.21.15.159:0/2318742583 >> 172.21.15.159:0/2318742583 conn(0x7f3998102c10 msgr2=0x7f3998105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:56.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.508+0000 7f399e000640 1 -- 172.21.15.159:0/2318742583 shutdown_connections 2024-09-19T07:22:56.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.508+0000 7f399e000640 1 -- 172.21.15.159:0/2318742583 wait complete. 2024-09-19T07:22:56.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.509+0000 7f399e000640 1 Processor -- start 2024-09-19T07:22:56.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.509+0000 7f399e000640 1 -- start start 2024-09-19T07:22:56.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.509+0000 7f399e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3998107440 0x7f399819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:56.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.509+0000 7f399e000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f399810b9f0 con 0x7f3998107440 2024-09-19T07:22:56.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.509+0000 7f399cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3998107440 0x7f399819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:56.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.510+0000 7f399cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3998107440 0x7f399819b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52740/0 (socket says 172.21.15.159:52740) 2024-09-19T07:22:56.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.510+0000 7f399cc00640 1 -- 172.21.15.159:0/4255081318 learned_addr learned my addr 172.21.15.159:0/4255081318 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:56.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.510+0000 7f399cc00640 1 -- 172.21.15.159:0/4255081318 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399819ba20 con 0x7f3998107440 2024-09-19T07:22:56.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.510+0000 7f399cc00640 1 --2- 172.21.15.159:0/4255081318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3998107440 0x7f399819b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f39880097f0 tx=0x7f3988030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:56.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.510+0000 7f3996000640 1 -- 172.21.15.159:0/4255081318 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3988042070 con 0x7f3998107440 2024-09-19T07:22:56.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.510+0000 7f3996000640 1 -- 172.21.15.159:0/4255081318 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3988032ca0 con 0x7f3998107440 2024-09-19T07:22:56.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.510+0000 7f399e000640 1 -- 172.21.15.159:0/4255081318 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f399819bcb0 con 0x7f3998107440 2024-09-19T07:22:56.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.510+0000 7f3996000640 1 -- 172.21.15.159:0/4255081318 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3988037040 con 0x7f3998107440 2024-09-19T07:22:56.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.511+0000 7f399e000640 1 -- 172.21.15.159:0/4255081318 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f399819e9a0 con 0x7f3998107440 2024-09-19T07:22:56.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.511+0000 7f3996000640 1 -- 172.21.15.159:0/4255081318 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f3988045070 con 0x7f3998107440 2024-09-19T07:22:56.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.512+0000 7f399e000640 1 -- 172.21.15.159:0/4255081318 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f399810bb70 con 0x7f3998107440 2024-09-19T07:22:56.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.512+0000 7f3996000640 1 --2- 172.21.15.159:0/4255081318 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f397c039160 0x7f397c03b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:56.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.512+0000 7f3996000640 1 -- 172.21.15.159:0/4255081318 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f398806c150 con 0x7f3998107440 2024-09-19T07:22:56.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.512+0000 7f398fe00640 1 --2- 172.21.15.159:0/4255081318 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f397c039160 0x7f397c03b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:56.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.513+0000 7f398fe00640 1 --2- 172.21.15.159:0/4255081318 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f397c039160 0x7f397c03b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f39800098c0 tx=0x7f3980006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:56.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.517+0000 7f3996000640 1 -- 172.21.15.159:0/4255081318 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f398804b170 con 0x7f3998107440 2024-09-19T07:22:56.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.694+0000 7f399e000640 1 -- 172.21.15.159:0/4255081318 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f39981078c0 con 0x7f3998107440 2024-09-19T07:22:56.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.695+0000 7f3996000640 1 -- 172.21.15.159:0/4255081318 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f398806bb70 con 0x7f3998107440 2024-09-19T07:22:56.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.698+0000 7f399e000640 1 -- 172.21.15.159:0/4255081318 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f397c039160 msgr2=0x7f397c03b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:56.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.698+0000 7f399e000640 1 --2- 172.21.15.159:0/4255081318 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f397c039160 0x7f397c03b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f39800098c0 tx=0x7f3980006db0 comp rx=0 tx=0).stop 2024-09-19T07:22:56.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.698+0000 7f399e000640 1 -- 172.21.15.159:0/4255081318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3998107440 msgr2=0x7f399819b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:56.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.698+0000 7f399e000640 1 --2- 172.21.15.159:0/4255081318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3998107440 0x7f399819b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f39880097f0 tx=0x7f3988030ec0 comp rx=0 tx=0).stop 2024-09-19T07:22:56.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.698+0000 7f399e000640 1 -- 172.21.15.159:0/4255081318 shutdown_connections 2024-09-19T07:22:56.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.698+0000 7f399e000640 1 --2- 172.21.15.159:0/4255081318 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f397c039160 0x7f397c03b620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:56.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.698+0000 7f399e000640 1 --2- 172.21.15.159:0/4255081318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3998107440 0x7f399819b4e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:56.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.698+0000 7f399e000640 1 -- 172.21.15.159:0/4255081318 >> 172.21.15.159:0/4255081318 conn(0x7f3998102c10 msgr2=0x7f39981037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:56.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.698+0000 7f399e000640 1 -- 172.21.15.159:0/4255081318 shutdown_connections 2024-09-19T07:22:56.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:55.698+0000 7f399e000640 1 -- 172.21.15.159:0/4255081318 wait complete. 2024-09-19T07:22:56.391 INFO:teuthology.orchestra.run.smithi159.stdout:Enabling cephadm module... 2024-09-19T07:22:57.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:57 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2209992142' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T07:22:57.387 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:57 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: ignoring --setuser ceph since I am not root 2024-09-19T07:22:57.387 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:57 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: ignoring --setgroup ceph since I am not root 2024-09-19T07:22:57.657 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:57 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:57.402+0000 7f26c9a2d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T07:22:57.657 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:57 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:57.473+0000 7f26c9a2d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T07:22:57.657 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:57 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:57.657+0000 7f26c9a2d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T07:22:57.879 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.887+0000 7f31a2400640 1 Processor -- start 2024-09-19T07:22:57.880 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.887+0000 7f31a2400640 1 -- start start 2024-09-19T07:22:57.880 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.888+0000 7f31a2400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f319c1076a0 0x7f319c107aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:57.880 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.888+0000 7f31a2400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f319c108070 con 0x7f319c1076a0 2024-09-19T07:22:57.880 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.888+0000 7f319b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f319c1076a0 0x7f319c107aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:57.880 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.888+0000 7f319b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f319c1076a0 0x7f319c107aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52744/0 (socket says 172.21.15.159:52744) 2024-09-19T07:22:57.880 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.888+0000 7f319b400640 1 -- 172.21.15.159:0/3787306975 learned_addr learned my addr 172.21.15.159:0/3787306975 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:57.880 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.888+0000 7f319b400640 1 -- 172.21.15.159:0/3787306975 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c108800 con 0x7f319c1076a0 2024-09-19T07:22:57.880 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.889+0000 7f319b400640 1 --2- 172.21.15.159:0/3787306975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f319c1076a0 0x7f319c107aa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f318c009e80 tx=0x7f318c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfde308c491553d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:57.880 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.890+0000 7f319aa00640 1 -- 172.21.15.159:0/3787306975 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f318c036070 con 0x7f319c1076a0 2024-09-19T07:22:57.880 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.890+0000 7f319aa00640 1 -- 172.21.15.159:0/3787306975 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f318c0318c0 con 0x7f319c1076a0 2024-09-19T07:22:57.881 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.890+0000 7f319aa00640 1 -- 172.21.15.159:0/3787306975 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f318c007ab0 con 0x7f319c1076a0 2024-09-19T07:22:57.881 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.890+0000 7f31a2400640 1 -- 172.21.15.159:0/3787306975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f319c1076a0 msgr2=0x7f319c107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:57.881 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.890+0000 7f31a2400640 1 --2- 172.21.15.159:0/3787306975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f319c1076a0 0x7f319c107aa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f318c009e80 tx=0x7f318c02a7a0 comp rx=0 tx=0).stop 2024-09-19T07:22:57.881 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.890+0000 7f31a2400640 1 -- 172.21.15.159:0/3787306975 shutdown_connections 2024-09-19T07:22:57.881 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.890+0000 7f31a2400640 1 --2- 172.21.15.159:0/3787306975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f319c1076a0 0x7f319c107aa0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:57.881 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.890+0000 7f31a2400640 1 -- 172.21.15.159:0/3787306975 >> 172.21.15.159:0/3787306975 conn(0x7f319c102e50 msgr2=0x7f319c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:57.881 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.890+0000 7f31a2400640 1 -- 172.21.15.159:0/3787306975 shutdown_connections 2024-09-19T07:22:57.881 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.891+0000 7f31a2400640 1 -- 172.21.15.159:0/3787306975 wait complete. 2024-09-19T07:22:57.881 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.891+0000 7f31a2400640 1 Processor -- start 2024-09-19T07:22:57.881 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.892+0000 7f31a2400640 1 -- start start 2024-09-19T07:22:57.882 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.892+0000 7f31a2400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f319c1076a0 0x7f319c07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:57.882 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.892+0000 7f31a2400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f319c10bbb0 con 0x7f319c1076a0 2024-09-19T07:22:57.882 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.892+0000 7f319b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f319c1076a0 0x7f319c07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:57.882 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.892+0000 7f319b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f319c1076a0 0x7f319c07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52750/0 (socket says 172.21.15.159:52750) 2024-09-19T07:22:57.882 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.892+0000 7f319b400640 1 -- 172.21.15.159:0/2209992142 learned_addr learned my addr 172.21.15.159:0/2209992142 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:57.882 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.892+0000 7f319b400640 1 -- 172.21.15.159:0/2209992142 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c07e0b0 con 0x7f319c1076a0 2024-09-19T07:22:57.882 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.893+0000 7f319b400640 1 --2- 172.21.15.159:0/2209992142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f319c1076a0 0x7f319c07db70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f318c02acc0 tx=0x7f318c031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:57.882 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.893+0000 7f3198c00640 1 -- 172.21.15.159:0/2209992142 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f318c036050 con 0x7f319c1076a0 2024-09-19T07:22:57.882 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.893+0000 7f3198c00640 1 -- 172.21.15.159:0/2209992142 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f318c037040 con 0x7f319c1076a0 2024-09-19T07:22:57.882 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.893+0000 7f31a2400640 1 -- 172.21.15.159:0/2209992142 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319c07c2a0 con 0x7f319c1076a0 2024-09-19T07:22:57.882 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.893+0000 7f3198c00640 1 -- 172.21.15.159:0/2209992142 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f318c03d890 con 0x7f319c1076a0 2024-09-19T07:22:57.882 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.893+0000 7f31a2400640 1 -- 172.21.15.159:0/2209992142 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319c07c7e0 con 0x7f319c1076a0 2024-09-19T07:22:57.882 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.894+0000 7f3198c00640 1 -- 172.21.15.159:0/2209992142 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f318c044020 con 0x7f319c1076a0 2024-09-19T07:22:57.882 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.894+0000 7f31a2400640 1 -- 172.21.15.159:0/2209992142 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f319c107b20 con 0x7f319c1076a0 2024-09-19T07:22:57.883 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.895+0000 7f3198c00640 1 --2- 172.21.15.159:0/2209992142 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f3180039160 0x7f318003b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:57.883 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.895+0000 7f3198c00640 1 -- 172.21.15.159:0/2209992142 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f318c042020 con 0x7f319c1076a0 2024-09-19T07:22:57.883 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.895+0000 7f3192a00640 1 --2- 172.21.15.159:0/2209992142 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f3180039160 0x7f318003b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:57.883 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.895+0000 7f3192a00640 1 --2- 172.21.15.159:0/2209992142 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f3180039160 0x7f318003b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f31840098c0 tx=0x7f3184006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:57.883 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:56.899+0000 7f3198c00640 1 -- 172.21.15.159:0/2209992142 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f318c04b020 con 0x7f319c1076a0 2024-09-19T07:22:57.883 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:57.118+0000 7f31a2400640 1 -- 172.21.15.159:0/2209992142 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f319c07ca60 con 0x7f319c1076a0 2024-09-19T07:22:57.883 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:57.170+0000 7f3198c00640 1 -- 172.21.15.159:0/2209992142 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f318c041c10 con 0x7f319c1076a0 2024-09-19T07:22:57.883 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:57.172+0000 7f3198c00640 1 -- 172.21.15.159:0/2209992142 <== mon.0 v2:172.21.15.159:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f318c06ec10 con 0x7f319c1076a0 2024-09-19T07:22:57.883 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:57.173+0000 7f31a2400640 1 -- 172.21.15.159:0/2209992142 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f3180039160 msgr2=0x7f318003b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:57.883 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:57.173+0000 7f31a2400640 1 --2- 172.21.15.159:0/2209992142 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f3180039160 0x7f318003b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f31840098c0 tx=0x7f3184006db0 comp rx=0 tx=0).stop 2024-09-19T07:22:57.883 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:57.173+0000 7f31a2400640 1 -- 172.21.15.159:0/2209992142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f319c1076a0 msgr2=0x7f319c07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:57.883 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:57.173+0000 7f31a2400640 1 --2- 172.21.15.159:0/2209992142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f319c1076a0 0x7f319c07db70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f318c02acc0 tx=0x7f318c031ca0 comp rx=0 tx=0).stop 2024-09-19T07:22:57.884 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:57.174+0000 7f31a2400640 1 -- 172.21.15.159:0/2209992142 shutdown_connections 2024-09-19T07:22:57.884 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:57.174+0000 7f31a2400640 1 --2- 172.21.15.159:0/2209992142 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f3180039160 0x7f318003b620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:57.884 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:57.174+0000 7f31a2400640 1 --2- 172.21.15.159:0/2209992142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f319c1076a0 0x7f319c07db70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:57.884 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:57.174+0000 7f31a2400640 1 -- 172.21.15.159:0/2209992142 >> 172.21.15.159:0/2209992142 conn(0x7f319c102e50 msgr2=0x7f319c1039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:57.884 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:57.174+0000 7f31a2400640 1 -- 172.21.15.159:0/2209992142 shutdown_connections 2024-09-19T07:22:57.884 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:57.174+0000 7f31a2400640 1 -- 172.21.15.159:0/2209992142 wait complete. 2024-09-19T07:22:57.915 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:57 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:57.738+0000 7f26c9a2d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T07:22:58.175 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:57 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:57.915+0000 7f26c9a2d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:22:58.175 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:58 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:58.097+0000 7f26c9a2d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T07:22:58.471 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:58 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2209992142' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T07:22:58.471 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:58 smithi159 ceph-mon[30472]: mgrmap e5: x(active, since 11s) 2024-09-19T07:22:58.728 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:58 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:58.485+0000 7f26c9a2d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T07:22:58.728 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:58 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:58.561+0000 7f26c9a2d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T07:22:59.055 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:58 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:58.791+0000 7f26c9a2d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T07:22:59.374 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:22:59 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2083166646' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T07:22:59.386 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout { 2024-09-19T07:22:59.386 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-19T07:22:59.386 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T07:22:59.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T07:22:59.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T07:22:59.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout } 2024-09-19T07:22:59.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.488+0000 7f64c4200640 1 Processor -- start 2024-09-19T07:22:59.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.488+0000 7f64c4200640 1 -- start start 2024-09-19T07:22:59.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.488+0000 7f64c4200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64bc1098c0 0x7f64bc109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:59.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.488+0000 7f64c4200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc10a290 con 0x7f64bc1098c0 2024-09-19T07:22:59.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.489+0000 7f64c1800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64bc1098c0 0x7f64bc109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:59.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.489+0000 7f64c1800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64bc1098c0 0x7f64bc109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52770/0 (socket says 172.21.15.159:52770) 2024-09-19T07:22:59.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.489+0000 7f64c1800640 1 -- 172.21.15.159:0/26134277 learned_addr learned my addr 172.21.15.159:0/26134277 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:59.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.489+0000 7f64c1800640 1 -- 172.21.15.159:0/26134277 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc10aa70 con 0x7f64bc1098c0 2024-09-19T07:22:59.387 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.490+0000 7f64c1800640 1 --2- 172.21.15.159:0/26134277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64bc1098c0 0x7f64bc109cc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009820 tx=0x7f64b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f031e5e2fd17e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:59.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.490+0000 7f64bbe00640 1 -- 172.21.15.159:0/26134277 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64b0037070 con 0x7f64bc1098c0 2024-09-19T07:22:59.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.490+0000 7f64bbe00640 1 -- 172.21.15.159:0/26134277 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f64b0032900 con 0x7f64bc1098c0 2024-09-19T07:22:59.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.490+0000 7f64bbe00640 1 -- 172.21.15.159:0/26134277 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64b0030850 con 0x7f64bc1098c0 2024-09-19T07:22:59.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.491+0000 7f64c4200640 1 -- 172.21.15.159:0/26134277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64bc1098c0 msgr2=0x7f64bc109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:59.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.491+0000 7f64c4200640 1 --2- 172.21.15.159:0/26134277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64bc1098c0 0x7f64bc109cc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009820 tx=0x7f64b002a270 comp rx=0 tx=0).stop 2024-09-19T07:22:59.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.491+0000 7f64c4200640 1 -- 172.21.15.159:0/26134277 shutdown_connections 2024-09-19T07:22:59.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.491+0000 7f64c4200640 1 --2- 172.21.15.159:0/26134277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64bc1098c0 0x7f64bc109cc0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:59.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.491+0000 7f64c4200640 1 -- 172.21.15.159:0/26134277 >> 172.21.15.159:0/26134277 conn(0x7f64bc07cf40 msgr2=0x7f64bc07d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:59.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.492+0000 7f64c4200640 1 -- 172.21.15.159:0/26134277 shutdown_connections 2024-09-19T07:22:59.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.492+0000 7f64c4200640 1 -- 172.21.15.159:0/26134277 wait complete. 2024-09-19T07:22:59.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.493+0000 7f64c4200640 1 Processor -- start 2024-09-19T07:22:59.388 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.493+0000 7f64c4200640 1 -- start start 2024-09-19T07:22:59.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.493+0000 7f64c4200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64bc1098c0 0x7f64bc19d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:59.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.493+0000 7f64c4200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc10de20 con 0x7f64bc1098c0 2024-09-19T07:22:59.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.493+0000 7f64c1800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64bc1098c0 0x7f64bc19d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:22:59.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.493+0000 7f64c1800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64bc1098c0 0x7f64bc19d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52772/0 (socket says 172.21.15.159:52772) 2024-09-19T07:22:59.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.493+0000 7f64c1800640 1 -- 172.21.15.159:0/2083166646 learned_addr learned my addr 172.21.15.159:0/2083166646 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:22:59.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.494+0000 7f64c1800640 1 -- 172.21.15.159:0/2083166646 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc19ddd0 con 0x7f64bc1098c0 2024-09-19T07:22:59.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.494+0000 7f64c1800640 1 --2- 172.21.15.159:0/2083166646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64bc1098c0 0x7f64bc19d890 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009940 tx=0x7f64b0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:22:59.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.494+0000 7f64ba000640 1 -- 172.21.15.159:0/2083166646 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64b0042070 con 0x7f64bc1098c0 2024-09-19T07:22:59.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.494+0000 7f64ba000640 1 -- 172.21.15.159:0/2083166646 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f64b00308c0 con 0x7f64bc1098c0 2024-09-19T07:22:59.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.494+0000 7f64ba000640 1 -- 172.21.15.159:0/2083166646 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64b0037040 con 0x7f64bc1098c0 2024-09-19T07:22:59.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.494+0000 7f64c4200640 1 -- 172.21.15.159:0/2083166646 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64bc19e060 con 0x7f64bc1098c0 2024-09-19T07:22:59.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.495+0000 7f64c4200640 1 -- 172.21.15.159:0/2083166646 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64bc1a0d50 con 0x7f64bc1098c0 2024-09-19T07:22:59.389 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.496+0000 7f64c4200640 1 -- 172.21.15.159:0/2083166646 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64bc109d40 con 0x7f64bc1098c0 2024-09-19T07:22:59.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.497+0000 7f64ba000640 1 -- 172.21.15.159:0/2083166646 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f64b0045070 con 0x7f64bc1098c0 2024-09-19T07:22:59.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.497+0000 7f64ba000640 1 --2- 172.21.15.159:0/2083166646 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f64a4039200 0x7f64a403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:22:59.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.497+0000 7f64ba000640 1 -- 172.21.15.159:0/2083166646 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f64b006c250 con 0x7f64bc1098c0 2024-09-19T07:22:59.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.497+0000 7f64c0e00640 1 -- 172.21.15.159:0/2083166646 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f64a4039200 msgr2=0x7f64a403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.159:6800/432893324 2024-09-19T07:22:59.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.497+0000 7f64c0e00640 1 --2- 172.21.15.159:0/2083166646 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f64a4039200 0x7f64a403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:22:59.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.502+0000 7f64ba000640 1 -- 172.21.15.159:0/2083166646 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f64b004b170 con 0x7f64bc1098c0 2024-09-19T07:22:59.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.698+0000 7f64c0e00640 1 -- 172.21.15.159:0/2083166646 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f64a4039200 msgr2=0x7f64a403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.159:6800/432893324 2024-09-19T07:22:59.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.698+0000 7f64c0e00640 1 --2- 172.21.15.159:0/2083166646 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f64a4039200 0x7f64a403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T07:22:59.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.728+0000 7f64c4200640 1 -- 172.21.15.159:0/2083166646 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f64bc1a1330 con 0x7f64bc1098c0 2024-09-19T07:22:59.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.728+0000 7f64ba000640 1 -- 172.21.15.159:0/2083166646 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f64b00327b0 con 0x7f64bc1098c0 2024-09-19T07:22:59.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.732+0000 7f64c4200640 1 -- 172.21.15.159:0/2083166646 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f64a4039200 msgr2=0x7f64a403b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:22:59.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.732+0000 7f64c4200640 1 --2- 172.21.15.159:0/2083166646 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f64a4039200 0x7f64a403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:59.390 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.732+0000 7f64c4200640 1 -- 172.21.15.159:0/2083166646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64bc1098c0 msgr2=0x7f64bc19d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:22:59.391 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.732+0000 7f64c4200640 1 --2- 172.21.15.159:0/2083166646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64bc1098c0 0x7f64bc19d890 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009940 tx=0x7f64b0032ce0 comp rx=0 tx=0).stop 2024-09-19T07:22:59.391 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.732+0000 7f64c4200640 1 -- 172.21.15.159:0/2083166646 shutdown_connections 2024-09-19T07:22:59.391 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.732+0000 7f64c4200640 1 --2- 172.21.15.159:0/2083166646 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f64a4039200 0x7f64a403b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:59.391 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.732+0000 7f64c4200640 1 --2- 172.21.15.159:0/2083166646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64bc1098c0 0x7f64bc19d890 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:22:59.391 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.732+0000 7f64c4200640 1 -- 172.21.15.159:0/2083166646 >> 172.21.15.159:0/2083166646 conn(0x7f64bc07cf40 msgr2=0x7f64bc107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:22:59.391 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.733+0000 7f64c4200640 1 -- 172.21.15.159:0/2083166646 shutdown_connections 2024-09-19T07:22:59.391 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:58.733+0000 7f64c4200640 1 -- 172.21.15.159:0/2083166646 wait complete. 2024-09-19T07:22:59.391 INFO:teuthology.orchestra.run.smithi159.stdout:Waiting for the mgr to restart... 2024-09-19T07:22:59.391 INFO:teuthology.orchestra.run.smithi159.stdout:Waiting for mgr epoch 5... 2024-09-19T07:22:59.979 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:59 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:59.911+0000 7f26c9a2d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T07:23:00.387 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:22:59 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:22:59.996+0000 7f26c9a2d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T07:23:01.198 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:00 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:00.899+0000 7f26c9a2d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T07:23:01.198 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:01 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:01.036+0000 7f26c9a2d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T07:23:01.198 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:01 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:01.196+0000 7f26c9a2d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:23:01.518 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:01 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:01.268+0000 7f26c9a2d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T07:23:01.518 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:01 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:01.436+0000 7f26c9a2d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T07:23:01.518 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:01 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:01.516+0000 7f26c9a2d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T07:23:01.786 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:01 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:01.596+0000 7f26c9a2d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T07:23:01.786 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:01 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:01.785+0000 7f26c9a2d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T07:23:02.037 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:01 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:01.862+0000 7f26c9a2d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T07:23:02.037 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:01 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:01.940+0000 7f26c9a2d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T07:23:02.388 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:02 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:02.036+0000 7f26c9a2d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T07:23:02.389 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:02 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:02.106+0000 7f26c9a2d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T07:23:02.389 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:02 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:02.178+0000 7f26c9a2d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T07:23:03.386 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:03 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:03.086+0000 7f26c9a2d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T07:23:03.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:03 smithi159 ceph-mon[30472]: Active manager daemon x restarted 2024-09-19T07:23:03.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:03 smithi159 ceph-mon[30472]: Activating manager daemon x 2024-09-19T07:23:03.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:03 smithi159 ceph-mon[30472]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T07:23:03.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:03 smithi159 ceph-mon[30472]: mgrmap e6: x(active, starting, since 0.0063238s) 2024-09-19T07:23:03.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:03 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:23:03.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:03 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T07:23:03.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:03 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T07:23:03.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:03 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T07:23:03.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:03 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T07:23:03.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:03 smithi159 ceph-mon[30472]: Manager daemon x is now available 2024-09-19T07:23:03.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:03 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' 2024-09-19T07:23:03.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:03 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' 2024-09-19T07:23:03.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:03 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:23:04.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:04 smithi159 ceph-mon[30472]: Found migration_current of "None". Setting to last migration. 2024-09-19T07:23:04.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:04 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:23:04.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:04 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T07:23:04.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:04 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T07:23:04.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:04 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' 2024-09-19T07:23:04.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:04 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' 2024-09-19T07:23:04.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:04 smithi159 ceph-mon[30472]: mgrmap e7: x(active, since 1.00671s) 2024-09-19T07:23:04.795 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout { 2024-09-19T07:23:04.796 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-19T07:23:04.796 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T07:23:04.796 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout } 2024-09-19T07:23:04.796 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.979+0000 7f0800e00640 1 Processor -- start 2024-09-19T07:23:04.796 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.979+0000 7f0800e00640 1 -- start start 2024-09-19T07:23:04.796 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.979+0000 7f0800e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc1076e0 0x7f07fc107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:04.796 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.979+0000 7f0800e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc1080b0 con 0x7f07fc1076e0 2024-09-19T07:23:04.796 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.980+0000 7f07f9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc1076e0 0x7f07fc107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:04.796 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.980+0000 7f07f9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc1076e0 0x7f07fc107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52784/0 (socket says 172.21.15.159:52784) 2024-09-19T07:23:04.796 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.980+0000 7f07f9e00640 1 -- 172.21.15.159:0/1283913027 learned_addr learned my addr 172.21.15.159:0/1283913027 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:04.797 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.980+0000 7f07f9e00640 1 -- 172.21.15.159:0/1283913027 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07fc108840 con 0x7f07fc1076e0 2024-09-19T07:23:04.797 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.981+0000 7f07f9e00640 1 --2- 172.21.15.159:0/1283913027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc1076e0 0x7f07fc107ae0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009820 tx=0x7f07ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f609f452b6eae781 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:04.797 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.981+0000 7f07f8a00640 1 -- 172.21.15.159:0/1283913027 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07ec037070 con 0x7f07fc1076e0 2024-09-19T07:23:04.797 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.981+0000 7f07f8a00640 1 -- 172.21.15.159:0/1283913027 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f07ec032900 con 0x7f07fc1076e0 2024-09-19T07:23:04.797 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.981+0000 7f07f8a00640 1 -- 172.21.15.159:0/1283913027 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07ec030850 con 0x7f07fc1076e0 2024-09-19T07:23:04.797 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.982+0000 7f0800e00640 1 -- 172.21.15.159:0/1283913027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc1076e0 msgr2=0x7f07fc107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:04.797 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.982+0000 7f0800e00640 1 --2- 172.21.15.159:0/1283913027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc1076e0 0x7f07fc107ae0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009820 tx=0x7f07ec02a270 comp rx=0 tx=0).stop 2024-09-19T07:23:04.797 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.982+0000 7f0800e00640 1 -- 172.21.15.159:0/1283913027 shutdown_connections 2024-09-19T07:23:04.797 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.982+0000 7f0800e00640 1 --2- 172.21.15.159:0/1283913027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc1076e0 0x7f07fc107ae0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:04.797 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.982+0000 7f0800e00640 1 -- 172.21.15.159:0/1283913027 >> 172.21.15.159:0/1283913027 conn(0x7f07fc102eb0 msgr2=0x7f07fc1052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:04.797 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.982+0000 7f0800e00640 1 -- 172.21.15.159:0/1283913027 shutdown_connections 2024-09-19T07:23:04.797 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.982+0000 7f0800e00640 1 -- 172.21.15.159:0/1283913027 wait complete. 2024-09-19T07:23:04.798 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.983+0000 7f0800e00640 1 Processor -- start 2024-09-19T07:23:04.798 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.983+0000 7f0800e00640 1 -- start start 2024-09-19T07:23:04.798 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.984+0000 7f0800e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc1076e0 0x7f07fc19d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:04.798 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.984+0000 7f0800e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc10bbf0 con 0x7f07fc1076e0 2024-09-19T07:23:04.798 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.984+0000 7f07f9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc1076e0 0x7f07fc19d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:04.798 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.984+0000 7f07f9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc1076e0 0x7f07fc19d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52798/0 (socket says 172.21.15.159:52798) 2024-09-19T07:23:04.798 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.984+0000 7f07f9e00640 1 -- 172.21.15.159:0/1462715699 learned_addr learned my addr 172.21.15.159:0/1462715699 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:04.798 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.984+0000 7f07f9e00640 1 -- 172.21.15.159:0/1462715699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07fc19de20 con 0x7f07fc1076e0 2024-09-19T07:23:04.798 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.985+0000 7f07f9e00640 1 --2- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc1076e0 0x7f07fc19d8e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0097f0 tx=0x7f07ec030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:04.798 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.985+0000 7f07f2a00640 1 -- 172.21.15.159:0/1462715699 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07ec042070 con 0x7f07fc1076e0 2024-09-19T07:23:04.798 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.985+0000 7f07f2a00640 1 -- 172.21.15.159:0/1462715699 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f07ec032ca0 con 0x7f07fc1076e0 2024-09-19T07:23:04.798 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.985+0000 7f07f2a00640 1 -- 172.21.15.159:0/1462715699 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07ec037040 con 0x7f07fc1076e0 2024-09-19T07:23:04.798 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.985+0000 7f0800e00640 1 -- 172.21.15.159:0/1462715699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07fc19e0b0 con 0x7f07fc1076e0 2024-09-19T07:23:04.799 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.985+0000 7f0800e00640 1 -- 172.21.15.159:0/1462715699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07fc1a0da0 con 0x7f07fc1076e0 2024-09-19T07:23:04.799 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.986+0000 7f07f2a00640 1 -- 172.21.15.159:0/1462715699 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f07ec045070 con 0x7f07fc1076e0 2024-09-19T07:23:04.799 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.987+0000 7f07f2a00640 1 --2- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f07e0039200 0x7f07e003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:04.799 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.987+0000 7f07f2a00640 1 -- 172.21.15.159:0/1462715699 --> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f07e003bdd0 con 0x7f07e0039200 2024-09-19T07:23:04.799 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.987+0000 7f07f2a00640 1 -- 172.21.15.159:0/1462715699 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f07ec06c0c0 con 0x7f07fc1076e0 2024-09-19T07:23:04.799 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.987+0000 7f07f9400640 1 -- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f07e0039200 msgr2=0x7f07e003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.159:6800/432893324 2024-09-19T07:23:04.799 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:22:59.987+0000 7f07f9400640 1 --2- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f07e0039200 0x7f07e003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:23:04.799 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:00.187+0000 7f07f9400640 1 -- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f07e0039200 msgr2=0x7f07e003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.159:6800/432893324 2024-09-19T07:23:04.799 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:00.187+0000 7f07f9400640 1 --2- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f07e0039200 0x7f07e003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T07:23:04.799 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:00.588+0000 7f07f9400640 1 -- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f07e0039200 msgr2=0x7f07e003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.159:6800/432893324 2024-09-19T07:23:04.799 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:00.588+0000 7f07f9400640 1 --2- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f07e0039200 0x7f07e003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T07:23:04.799 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:01.389+0000 7f07f9400640 1 -- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f07e0039200 msgr2=0x7f07e003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.159:6800/432893324 2024-09-19T07:23:04.799 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:01.389+0000 7f07f9400640 1 --2- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f07e0039200 0x7f07e003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T07:23:04.800 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:02.991+0000 7f07f9400640 1 -- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f07e0039200 msgr2=0x7f07e003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.159:6800/432893324 2024-09-19T07:23:04.800 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:02.991+0000 7f07f9400640 1 --2- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f07e0039200 0x7f07e003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-19T07:23:04.800 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:03.093+0000 7f07f2a00640 1 -- 172.21.15.159:0/1462715699 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f07ec03da60 con 0x7f07fc1076e0 2024-09-19T07:23:04.800 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:03.093+0000 7f07f2a00640 1 -- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f07e0039200 msgr2=0x7f07e003b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:23:04.800 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:03.093+0000 7f07f2a00640 1 --2- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f07e0039200 0x7f07e003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:04.800 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.092+0000 7f07f2a00640 1 -- 172.21.15.159:0/1462715699 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f07ec06d810 con 0x7f07fc1076e0 2024-09-19T07:23:04.800 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.092+0000 7f07f2a00640 1 --2- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f07e003cb30 0x7f07e003ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:04.800 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.092+0000 7f07f2a00640 1 -- 172.21.15.159:0/1462715699 --> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f07ec048e50 con 0x7f07e003cb30 2024-09-19T07:23:04.800 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.093+0000 7f07f9400640 1 --2- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f07e003cb30 0x7f07e003ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:04.800 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.094+0000 7f07f9400640 1 --2- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f07e003cb30 0x7f07e003ef20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f07e8003980 tx=0x7f07e80092b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:04.800 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.095+0000 7f07f2a00640 1 -- 172.21.15.159:0/1462715699 <== mgr.14116 v2:172.21.15.159:6800/2966947684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f07ec048e50 con 0x7f07e003cb30 2024-09-19T07:23:04.800 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.102+0000 7f07e7e00640 1 -- 172.21.15.159:0/1462715699 --> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f07d4002570 con 0x7f07e003cb30 2024-09-19T07:23:04.800 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.103+0000 7f07f2a00640 1 -- 172.21.15.159:0/1462715699 <== mgr.14116 v2:172.21.15.159:6800/2966947684 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f07d4002570 con 0x7f07e003cb30 2024-09-19T07:23:04.801 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.103+0000 7f07e7e00640 1 -- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f07e003cb30 msgr2=0x7f07e003ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:04.801 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.103+0000 7f07e7e00640 1 --2- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f07e003cb30 0x7f07e003ef20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f07e8003980 tx=0x7f07e80092b0 comp rx=0 tx=0).stop 2024-09-19T07:23:04.801 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.103+0000 7f07e7e00640 1 -- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc1076e0 msgr2=0x7f07fc19d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:04.801 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.103+0000 7f07e7e00640 1 --2- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc1076e0 0x7f07fc19d8e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0097f0 tx=0x7f07ec030ec0 comp rx=0 tx=0).stop 2024-09-19T07:23:04.801 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.103+0000 7f07e7e00640 1 -- 172.21.15.159:0/1462715699 shutdown_connections 2024-09-19T07:23:04.801 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.103+0000 7f07e7e00640 1 --2- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f07e003cb30 0x7f07e003ef20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:04.801 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.103+0000 7f07e7e00640 1 --2- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:6800/432893324,v1:172.21.15.159:6801/432893324] conn(0x7f07e0039200 0x7f07e003b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:04.801 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.103+0000 7f07e7e00640 1 --2- 172.21.15.159:0/1462715699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc1076e0 0x7f07fc19d8e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:04.801 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.103+0000 7f07e7e00640 1 -- 172.21.15.159:0/1462715699 >> 172.21.15.159:0/1462715699 conn(0x7f07fc102eb0 msgr2=0x7f07fc1039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:04.801 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.104+0000 7f07e7e00640 1 -- 172.21.15.159:0/1462715699 shutdown_connections 2024-09-19T07:23:04.801 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:04.104+0000 7f07e7e00640 1 -- 172.21.15.159:0/1462715699 wait complete. 2024-09-19T07:23:04.801 INFO:teuthology.orchestra.run.smithi159.stdout:mgr epoch 5 is available 2024-09-19T07:23:04.801 INFO:teuthology.orchestra.run.smithi159.stdout:Setting orchestrator backend to cephadm... 2024-09-19T07:23:05.315 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-mon[30472]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T07:23:05.315 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-mon[30472]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T07:23:05.315 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:05.279+0000 7f2622000640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:07:23:05] ENGINE Error in HTTPServer.serve 2024-09-19T07:23:05.316 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: Traceback (most recent call last): 2024-09-19T07:23:05.316 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T07:23:05.316 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: self._connections.run(self.expiration_interval) 2024-09-19T07:23:05.316 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T07:23:05.316 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: self._run(expiration_interval) 2024-09-19T07:23:05.316 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T07:23:05.316 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T07:23:05.316 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T07:23:05.316 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T07:23:05.316 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T07:23:05.316 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: s = self.context.wrap_socket( 2024-09-19T07:23:05.316 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T07:23:05.316 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: return self.sslsocket_class._create( 2024-09-19T07:23:05.317 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T07:23:05.317 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: self.do_handshake() 2024-09-19T07:23:05.317 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T07:23:05.317 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: self._sslobj.do_handshake() 2024-09-19T07:23:05.317 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T07:23:05.317 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:06.255 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.308+0000 7f8d47200640 1 Processor -- start 2024-09-19T07:23:06.256 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.308+0000 7f8d47200640 1 -- start start 2024-09-19T07:23:06.256 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.309+0000 7f8d47200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40081640 0x7f8d40081a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:06.256 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.309+0000 7f8d47200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8d40082010 con 0x7f8d40081640 2024-09-19T07:23:06.256 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.309+0000 7f8d3fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40081640 0x7f8d40081a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:06.256 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.309+0000 7f8d3fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40081640 0x7f8d40081a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49800/0 (socket says 172.21.15.159:49800) 2024-09-19T07:23:06.256 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.309+0000 7f8d3fe00640 1 -- 172.21.15.159:0/133776318 learned_addr learned my addr 172.21.15.159:0/133776318 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:06.256 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.309+0000 7f8d3fe00640 1 -- 172.21.15.159:0/133776318 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4010ee10 con 0x7f8d40081640 2024-09-19T07:23:06.256 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.310+0000 7f8d3fe00640 1 --2- 172.21.15.159:0/133776318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40081640 0x7f8d40081a40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009820 tx=0x7f8d3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694d554182fcddb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:06.256 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.310+0000 7f8d3f400640 1 -- 172.21.15.159:0/133776318 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d30037070 con 0x7f8d40081640 2024-09-19T07:23:06.256 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.310+0000 7f8d3f400640 1 -- 172.21.15.159:0/133776318 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8d30032900 con 0x7f8d40081640 2024-09-19T07:23:06.256 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.310+0000 7f8d3f400640 1 -- 172.21.15.159:0/133776318 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d30030850 con 0x7f8d40081640 2024-09-19T07:23:06.257 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.311+0000 7f8d47200640 1 -- 172.21.15.159:0/133776318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40081640 msgr2=0x7f8d40081a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:06.257 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.311+0000 7f8d47200640 1 --2- 172.21.15.159:0/133776318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40081640 0x7f8d40081a40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009820 tx=0x7f8d3002a270 comp rx=0 tx=0).stop 2024-09-19T07:23:06.257 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.311+0000 7f8d47200640 1 -- 172.21.15.159:0/133776318 shutdown_connections 2024-09-19T07:23:06.257 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.311+0000 7f8d47200640 1 --2- 172.21.15.159:0/133776318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40081640 0x7f8d40081a40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:06.257 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.311+0000 7f8d47200640 1 -- 172.21.15.159:0/133776318 >> 172.21.15.159:0/133776318 conn(0x7f8d4007cdd0 msgr2=0x7f8d4007f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:06.257 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.311+0000 7f8d47200640 1 -- 172.21.15.159:0/133776318 shutdown_connections 2024-09-19T07:23:06.257 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.311+0000 7f8d47200640 1 -- 172.21.15.159:0/133776318 wait complete. 2024-09-19T07:23:06.257 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.312+0000 7f8d47200640 1 Processor -- start 2024-09-19T07:23:06.257 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.312+0000 7f8d47200640 1 -- start start 2024-09-19T07:23:06.257 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.312+0000 7f8d47200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40081640 0x7f8d401a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:06.257 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.312+0000 7f8d47200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8d401121c0 con 0x7f8d40081640 2024-09-19T07:23:06.257 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.313+0000 7f8d3fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40081640 0x7f8d401a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:06.257 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.313+0000 7f8d3fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40081640 0x7f8d401a1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49804/0 (socket says 172.21.15.159:49804) 2024-09-19T07:23:06.258 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.313+0000 7f8d3fe00640 1 -- 172.21.15.159:0/137454682 learned_addr learned my addr 172.21.15.159:0/137454682 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:06.258 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.313+0000 7f8d3fe00640 1 -- 172.21.15.159:0/137454682 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d401a21a0 con 0x7f8d40081640 2024-09-19T07:23:06.258 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.313+0000 7f8d3fe00640 1 --2- 172.21.15.159:0/137454682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40081640 0x7f8d401a1c60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8d300097f0 tx=0x7f8d30030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:06.258 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.313+0000 7f8d3d600640 1 -- 172.21.15.159:0/137454682 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d30037040 con 0x7f8d40081640 2024-09-19T07:23:06.258 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.314+0000 7f8d3d600640 1 -- 172.21.15.159:0/137454682 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8d30040070 con 0x7f8d40081640 2024-09-19T07:23:06.258 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.314+0000 7f8d3d600640 1 -- 172.21.15.159:0/137454682 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d3003a390 con 0x7f8d40081640 2024-09-19T07:23:06.258 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.314+0000 7f8d47200640 1 -- 172.21.15.159:0/137454682 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d401a2430 con 0x7f8d40081640 2024-09-19T07:23:06.258 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.314+0000 7f8d47200640 1 -- 172.21.15.159:0/137454682 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d401a5120 con 0x7f8d40081640 2024-09-19T07:23:06.258 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.315+0000 7f8d47200640 1 -- 172.21.15.159:0/137454682 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d40081ac0 con 0x7f8d40081640 2024-09-19T07:23:06.258 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.315+0000 7f8d3d600640 1 -- 172.21.15.159:0/137454682 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f8d3003b430 con 0x7f8d40081640 2024-09-19T07:23:06.258 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.316+0000 7f8d3d600640 1 --2- 172.21.15.159:0/137454682 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f8d24039200 0x7f8d2403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:06.258 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.316+0000 7f8d3d600640 1 -- 172.21.15.159:0/137454682 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8d3006c7d0 con 0x7f8d40081640 2024-09-19T07:23:06.259 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.316+0000 7f8d37400640 1 --2- 172.21.15.159:0/137454682 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f8d24039200 0x7f8d2403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:06.259 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.317+0000 7f8d37400640 1 --2- 172.21.15.159:0/137454682 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f8d24039200 0x7f8d2403b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8d280098c0 tx=0x7f8d28006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:06.259 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.321+0000 7f8d3d600640 1 -- 172.21.15.159:0/137454682 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8d3006c120 con 0x7f8d40081640 2024-09-19T07:23:06.259 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.515+0000 7f8d47200640 1 -- 172.21.15.159:0/137454682 --> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f8d4007e060 con 0x7f8d24039200 2024-09-19T07:23:06.259 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.522+0000 7f8d3d600640 1 -- 172.21.15.159:0/137454682 <== mgr.14116 v2:172.21.15.159:6800/2966947684 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8d4007e060 con 0x7f8d24039200 2024-09-19T07:23:06.259 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.524+0000 7f8d47200640 1 -- 172.21.15.159:0/137454682 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f8d24039200 msgr2=0x7f8d2403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:06.259 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.524+0000 7f8d47200640 1 --2- 172.21.15.159:0/137454682 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f8d24039200 0x7f8d2403b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8d280098c0 tx=0x7f8d28006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:06.259 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.524+0000 7f8d47200640 1 -- 172.21.15.159:0/137454682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40081640 msgr2=0x7f8d401a1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:06.259 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.524+0000 7f8d47200640 1 --2- 172.21.15.159:0/137454682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40081640 0x7f8d401a1c60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8d300097f0 tx=0x7f8d30030d80 comp rx=0 tx=0).stop 2024-09-19T07:23:06.259 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.524+0000 7f8d47200640 1 -- 172.21.15.159:0/137454682 shutdown_connections 2024-09-19T07:23:06.259 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.525+0000 7f8d47200640 1 --2- 172.21.15.159:0/137454682 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f8d24039200 0x7f8d2403b6c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:06.259 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.525+0000 7f8d47200640 1 --2- 172.21.15.159:0/137454682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d40081640 0x7f8d401a1c60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:06.259 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.525+0000 7f8d47200640 1 -- 172.21.15.159:0/137454682 >> 172.21.15.159:0/137454682 conn(0x7f8d4007cdd0 msgr2=0x7f8d4007d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:06.260 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.525+0000 7f8d47200640 1 -- 172.21.15.159:0/137454682 shutdown_connections 2024-09-19T07:23:06.260 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:05.525+0000 7f8d47200640 1 -- 172.21.15.159:0/137454682 wait complete. 2024-09-19T07:23:06.506 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:06 smithi159 ceph-mon[30472]: [19/Sep/2024:07:23:05] ENGINE Bus STARTING 2024-09-19T07:23:06.506 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:06 smithi159 ceph-mon[30472]: mgrmap e8: x(active, since 2s) 2024-09-19T07:23:06.506 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:06 smithi159 ceph-mon[30472]: [19/Sep/2024:07:23:05] ENGINE Serving on http://172.21.15.159:8765 2024-09-19T07:23:06.506 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:06 smithi159 ceph-mon[30472]: [19/Sep/2024:07:23:05] ENGINE Serving on https://172.21.15.159:7150 2024-09-19T07:23:06.506 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:06 smithi159 ceph-mon[30472]: [19/Sep/2024:07:23:05] ENGINE Bus STARTED 2024-09-19T07:23:06.506 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:06 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:23:06.507 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:06 smithi159 ceph-mon[30472]: [19/Sep/2024:07:23:05] ENGINE Error in HTTPServer.serve 2024-09-19T07:23:06.507 INFO:journalctl@ceph.mon.a.smithi159.stdout: Traceback (most recent call last): 2024-09-19T07:23:06.507 INFO:journalctl@ceph.mon.a.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T07:23:06.507 INFO:journalctl@ceph.mon.a.smithi159.stdout: self._connections.run(self.expiration_interval) 2024-09-19T07:23:06.507 INFO:journalctl@ceph.mon.a.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T07:23:06.507 INFO:journalctl@ceph.mon.a.smithi159.stdout: self._run(expiration_interval) 2024-09-19T07:23:06.507 INFO:journalctl@ceph.mon.a.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T07:23:06.507 INFO:journalctl@ceph.mon.a.smithi159.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T07:23:06.507 INFO:journalctl@ceph.mon.a.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T07:23:06.508 INFO:journalctl@ceph.mon.a.smithi159.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T07:23:06.508 INFO:journalctl@ceph.mon.a.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T07:23:06.508 INFO:journalctl@ceph.mon.a.smithi159.stdout: s = self.context.wrap_socket( 2024-09-19T07:23:06.508 INFO:journalctl@ceph.mon.a.smithi159.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T07:23:06.508 INFO:journalctl@ceph.mon.a.smithi159.stdout: return self.sslsocket_class._create( 2024-09-19T07:23:06.508 INFO:journalctl@ceph.mon.a.smithi159.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T07:23:06.508 INFO:journalctl@ceph.mon.a.smithi159.stdout: self.do_handshake() 2024-09-19T07:23:06.508 INFO:journalctl@ceph.mon.a.smithi159.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T07:23:06.508 INFO:journalctl@ceph.mon.a.smithi159.stdout: self._sslobj.do_handshake() 2024-09-19T07:23:06.508 INFO:journalctl@ceph.mon.a.smithi159.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T07:23:06.509 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:06 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' 2024-09-19T07:23:06.509 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:06 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:23:07.400 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:07 smithi159 ceph-mon[30472]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:23:07.721 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T07:23:07.722 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.753+0000 7f3761200640 1 Processor -- start 2024-09-19T07:23:07.722 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.754+0000 7f3761200640 1 -- start start 2024-09-19T07:23:07.722 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.754+0000 7f3761200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f375c109970 0x7f375c109d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:07.722 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.754+0000 7f3761200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f375c10a340 con 0x7f375c109970 2024-09-19T07:23:07.722 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.755+0000 7f375b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f375c109970 0x7f375c109d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:07.722 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.755+0000 7f375b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f375c109970 0x7f375c109d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49818/0 (socket says 172.21.15.159:49818) 2024-09-19T07:23:07.723 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.755+0000 7f375b400640 1 -- 172.21.15.159:0/3216552965 learned_addr learned my addr 172.21.15.159:0/3216552965 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:07.723 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.755+0000 7f375b400640 1 -- 172.21.15.159:0/3216552965 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375c10a840 con 0x7f375c109970 2024-09-19T07:23:07.723 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.756+0000 7f375b400640 1 --2- 172.21.15.159:0/3216552965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f375c109970 0x7f375c109d70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f374c009820 tx=0x7f374c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc90c639177305a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:07.723 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.756+0000 7f375a000640 1 -- 172.21.15.159:0/3216552965 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f374c037070 con 0x7f375c109970 2024-09-19T07:23:07.723 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.757+0000 7f375a000640 1 -- 172.21.15.159:0/3216552965 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f374c032900 con 0x7f375c109970 2024-09-19T07:23:07.723 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.757+0000 7f375a000640 1 -- 172.21.15.159:0/3216552965 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f374c030850 con 0x7f375c109970 2024-09-19T07:23:07.724 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.757+0000 7f3761200640 1 -- 172.21.15.159:0/3216552965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f375c109970 msgr2=0x7f375c109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:07.724 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.757+0000 7f3761200640 1 --2- 172.21.15.159:0/3216552965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f375c109970 0x7f375c109d70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f374c009820 tx=0x7f374c02a270 comp rx=0 tx=0).stop 2024-09-19T07:23:07.724 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.757+0000 7f3761200640 1 -- 172.21.15.159:0/3216552965 shutdown_connections 2024-09-19T07:23:07.724 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.757+0000 7f3761200640 1 --2- 172.21.15.159:0/3216552965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f375c109970 0x7f375c109d70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:07.724 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.757+0000 7f3761200640 1 -- 172.21.15.159:0/3216552965 >> 172.21.15.159:0/3216552965 conn(0x7f375c07ccd0 msgr2=0x7f375c07d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:07.724 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.757+0000 7f3761200640 1 -- 172.21.15.159:0/3216552965 shutdown_connections 2024-09-19T07:23:07.724 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.757+0000 7f3761200640 1 -- 172.21.15.159:0/3216552965 wait complete. 2024-09-19T07:23:07.724 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.758+0000 7f3761200640 1 Processor -- start 2024-09-19T07:23:07.725 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.758+0000 7f3761200640 1 -- start start 2024-09-19T07:23:07.725 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.759+0000 7f3761200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f375c109970 0x7f375c19d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:07.725 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.759+0000 7f3761200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f375c10dbf0 con 0x7f375c109970 2024-09-19T07:23:07.725 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.759+0000 7f375b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f375c109970 0x7f375c19d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:07.725 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.759+0000 7f375b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f375c109970 0x7f375c19d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49832/0 (socket says 172.21.15.159:49832) 2024-09-19T07:23:07.725 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.759+0000 7f375b400640 1 -- 172.21.15.159:0/1976031160 learned_addr learned my addr 172.21.15.159:0/1976031160 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:07.725 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.759+0000 7f375b400640 1 -- 172.21.15.159:0/1976031160 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375c19dc70 con 0x7f375c109970 2024-09-19T07:23:07.725 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.760+0000 7f375b400640 1 --2- 172.21.15.159:0/1976031160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f375c109970 0x7f375c19d730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f374c0097f0 tx=0x7f374c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:07.725 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.760+0000 7f3753e00640 1 -- 172.21.15.159:0/1976031160 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f374c042070 con 0x7f375c109970 2024-09-19T07:23:07.725 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.760+0000 7f3753e00640 1 -- 172.21.15.159:0/1976031160 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f374c032be0 con 0x7f375c109970 2024-09-19T07:23:07.725 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.760+0000 7f3753e00640 1 -- 172.21.15.159:0/1976031160 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f374c037040 con 0x7f375c109970 2024-09-19T07:23:07.726 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.760+0000 7f3761200640 1 -- 172.21.15.159:0/1976031160 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375c19df00 con 0x7f375c109970 2024-09-19T07:23:07.726 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.760+0000 7f3761200640 1 -- 172.21.15.159:0/1976031160 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f375c1a0bf0 con 0x7f375c109970 2024-09-19T07:23:07.726 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.761+0000 7f3753e00640 1 -- 172.21.15.159:0/1976031160 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f374c045070 con 0x7f375c109970 2024-09-19T07:23:07.726 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.761+0000 7f3761200640 1 -- 172.21.15.159:0/1976031160 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f375c109df0 con 0x7f375c109970 2024-09-19T07:23:07.726 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.761+0000 7f3753e00640 1 --2- 172.21.15.159:0/1976031160 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f3740039200 0x7f374003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:07.726 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.762+0000 7f3753e00640 1 -- 172.21.15.159:0/1976031160 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f374c06c180 con 0x7f375c109970 2024-09-19T07:23:07.726 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.762+0000 7f375aa00640 1 --2- 172.21.15.159:0/1976031160 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f3740039200 0x7f374003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:07.726 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.762+0000 7f375aa00640 1 --2- 172.21.15.159:0/1976031160 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f3740039200 0x7f374003b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f37480098c0 tx=0x7f3748006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:07.726 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.767+0000 7f3753e00640 1 -- 172.21.15.159:0/1976031160 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f374c043260 con 0x7f375c109970 2024-09-19T07:23:07.726 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.943+0000 7f3761200640 1 -- 172.21.15.159:0/1976031160 --> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f375c107d10 con 0x7f3740039200 2024-09-19T07:23:07.726 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.944+0000 7f3753e00640 1 -- 172.21.15.159:0/1976031160 <== mgr.14116 v2:172.21.15.159:6800/2966947684 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f375c107d10 con 0x7f3740039200 2024-09-19T07:23:07.726 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.946+0000 7f3761200640 1 -- 172.21.15.159:0/1976031160 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f3740039200 msgr2=0x7f374003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:07.727 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.946+0000 7f3761200640 1 --2- 172.21.15.159:0/1976031160 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f3740039200 0x7f374003b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f37480098c0 tx=0x7f3748006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:07.727 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.946+0000 7f3761200640 1 -- 172.21.15.159:0/1976031160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f375c109970 msgr2=0x7f375c19d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:07.727 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.946+0000 7f3761200640 1 --2- 172.21.15.159:0/1976031160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f375c109970 0x7f375c19d730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f374c0097f0 tx=0x7f374c030d80 comp rx=0 tx=0).stop 2024-09-19T07:23:07.727 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.946+0000 7f3761200640 1 -- 172.21.15.159:0/1976031160 shutdown_connections 2024-09-19T07:23:07.727 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.946+0000 7f3761200640 1 --2- 172.21.15.159:0/1976031160 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f3740039200 0x7f374003b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:07.727 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.946+0000 7f3761200640 1 --2- 172.21.15.159:0/1976031160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f375c109970 0x7f375c19d730 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:07.727 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.946+0000 7f3761200640 1 -- 172.21.15.159:0/1976031160 >> 172.21.15.159:0/1976031160 conn(0x7f375c07ccd0 msgr2=0x7f375c107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:07.727 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.946+0000 7f3761200640 1 -- 172.21.15.159:0/1976031160 shutdown_connections 2024-09-19T07:23:07.727 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:06.946+0000 7f3761200640 1 -- 172.21.15.159:0/1976031160 wait complete. 2024-09-19T07:23:07.727 INFO:teuthology.orchestra.run.smithi159.stdout:Generating ssh key... 2024-09-19T07:23:08.214 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-mon[30472]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:23:08.887 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: Generating public/private rsa key pair. 2024-09-19T07:23:08.887 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: Your identification has been saved in /tmp/tmpx8jh1q0t/key 2024-09-19T07:23:08.887 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: Your public key has been saved in /tmp/tmpx8jh1q0t/key.pub 2024-09-19T07:23:08.887 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: The key fingerprint is: 2024-09-19T07:23:08.887 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: SHA256:TgHYIuadoTHeYW5Y+ausuQYknmNbQT5Tik56LtT0HTI ceph-c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:23:08.887 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: The key's randomart image is: 2024-09-19T07:23:08.887 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: +---[RSA 3072]----+ 2024-09-19T07:23:08.888 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: | +. | 2024-09-19T07:23:08.888 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: | =.O... | 2024-09-19T07:23:08.888 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: | ++%o* . | 2024-09-19T07:23:08.888 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: |.+=B*E.. . | 2024-09-19T07:23:08.888 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: |B +.= +.S | 2024-09-19T07:23:08.888 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: |oB.o ..+ | 2024-09-19T07:23:08.888 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: |o++. . . | 2024-09-19T07:23:08.888 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: |..o.o | 2024-09-19T07:23:08.888 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: | o+o | 2024-09-19T07:23:08.888 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: +----[SHA256]-----+ 2024-09-19T07:23:09.227 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.200+0000 7f51bea00640 1 Processor -- start 2024-09-19T07:23:09.227 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.201+0000 7f51bea00640 1 -- start start 2024-09-19T07:23:09.228 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.201+0000 7f51bea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b807de80 0x7f51b807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:09.228 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.201+0000 7f51bea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f51b807c8b0 con 0x7f51b807de80 2024-09-19T07:23:09.228 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.201+0000 7f51bd600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b807de80 0x7f51b807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:09.228 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.201+0000 7f51bd600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b807de80 0x7f51b807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49836/0 (socket says 172.21.15.159:49836) 2024-09-19T07:23:09.228 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.201+0000 7f51bd600640 1 -- 172.21.15.159:0/1557934990 learned_addr learned my addr 172.21.15.159:0/1557934990 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:09.228 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.202+0000 7f51bd600640 1 -- 172.21.15.159:0/1557934990 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b807ca30 con 0x7f51b807de80 2024-09-19T07:23:09.228 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.203+0000 7f51bd600640 1 --2- 172.21.15.159:0/1557934990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b807de80 0x7f51b807c2e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009820 tx=0x7f51ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a2be49766ba4ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:09.228 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.203+0000 7f51b7e00640 1 -- 172.21.15.159:0/1557934990 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51ac037070 con 0x7f51b807de80 2024-09-19T07:23:09.229 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.203+0000 7f51b7e00640 1 -- 172.21.15.159:0/1557934990 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f51ac032900 con 0x7f51b807de80 2024-09-19T07:23:09.229 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.203+0000 7f51b7e00640 1 -- 172.21.15.159:0/1557934990 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51ac030850 con 0x7f51b807de80 2024-09-19T07:23:09.229 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.203+0000 7f51bea00640 1 -- 172.21.15.159:0/1557934990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b807de80 msgr2=0x7f51b807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:09.229 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.203+0000 7f51bea00640 1 --2- 172.21.15.159:0/1557934990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b807de80 0x7f51b807c2e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009820 tx=0x7f51ac02a270 comp rx=0 tx=0).stop 2024-09-19T07:23:09.229 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.204+0000 7f51bea00640 1 -- 172.21.15.159:0/1557934990 shutdown_connections 2024-09-19T07:23:09.229 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.204+0000 7f51bea00640 1 --2- 172.21.15.159:0/1557934990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b807de80 0x7f51b807c2e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:09.229 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.204+0000 7f51bea00640 1 -- 172.21.15.159:0/1557934990 >> 172.21.15.159:0/1557934990 conn(0x7f51b8102c10 msgr2=0x7f51b8105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:09.229 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.204+0000 7f51bea00640 1 -- 172.21.15.159:0/1557934990 shutdown_connections 2024-09-19T07:23:09.229 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.204+0000 7f51bea00640 1 -- 172.21.15.159:0/1557934990 wait complete. 2024-09-19T07:23:09.229 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.205+0000 7f51bea00640 1 Processor -- start 2024-09-19T07:23:09.229 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.205+0000 7f51bea00640 1 -- start start 2024-09-19T07:23:09.229 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.205+0000 7f51bea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b807de80 0x7f51b819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:09.229 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.205+0000 7f51bea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f51b810da70 con 0x7f51b807de80 2024-09-19T07:23:09.230 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.205+0000 7f51bd600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b807de80 0x7f51b819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:09.230 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.205+0000 7f51bd600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b807de80 0x7f51b819d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49842/0 (socket says 172.21.15.159:49842) 2024-09-19T07:23:09.230 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.205+0000 7f51bd600640 1 -- 172.21.15.159:0/643899917 learned_addr learned my addr 172.21.15.159:0/643899917 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:09.230 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.206+0000 7f51bd600640 1 -- 172.21.15.159:0/643899917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b819dbf0 con 0x7f51b807de80 2024-09-19T07:23:09.230 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.206+0000 7f51bd600640 1 --2- 172.21.15.159:0/643899917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b807de80 0x7f51b819d6b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0097f0 tx=0x7f51ac030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:09.230 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.206+0000 7f51b6000640 1 -- 172.21.15.159:0/643899917 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51ac042070 con 0x7f51b807de80 2024-09-19T07:23:09.230 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.206+0000 7f51b6000640 1 -- 172.21.15.159:0/643899917 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f51ac032ca0 con 0x7f51b807de80 2024-09-19T07:23:09.230 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.206+0000 7f51bea00640 1 -- 172.21.15.159:0/643899917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b819de80 con 0x7f51b807de80 2024-09-19T07:23:09.230 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.206+0000 7f51b6000640 1 -- 172.21.15.159:0/643899917 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51ac037040 con 0x7f51b807de80 2024-09-19T07:23:09.230 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.206+0000 7f51bea00640 1 -- 172.21.15.159:0/643899917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b81a0b70 con 0x7f51b807de80 2024-09-19T07:23:09.230 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.207+0000 7f51b6000640 1 -- 172.21.15.159:0/643899917 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f51ac045070 con 0x7f51b807de80 2024-09-19T07:23:09.230 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.207+0000 7f51bea00640 1 -- 172.21.15.159:0/643899917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b807c360 con 0x7f51b807de80 2024-09-19T07:23:09.230 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.208+0000 7f51b6000640 1 --2- 172.21.15.159:0/643899917 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f51a00391b0 0x7f51a003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:09.230 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.208+0000 7f51b6000640 1 -- 172.21.15.159:0/643899917 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f51ac06c390 con 0x7f51b807de80 2024-09-19T07:23:09.231 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.208+0000 7f51bcc00640 1 --2- 172.21.15.159:0/643899917 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f51a00391b0 0x7f51a003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:09.231 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.209+0000 7f51bcc00640 1 --2- 172.21.15.159:0/643899917 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f51a00391b0 0x7f51a003b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f51a40098c0 tx=0x7f51a4006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:09.231 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.213+0000 7f51b6000640 1 -- 172.21.15.159:0/643899917 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f51ac03d430 con 0x7f51b807de80 2024-09-19T07:23:09.231 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.392+0000 7f51bea00640 1 -- 172.21.15.159:0/643899917 --> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f51b8103f20 con 0x7f51a00391b0 2024-09-19T07:23:09.231 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.588+0000 7f51b6000640 1 -- 172.21.15.159:0/643899917 <== mgr.14116 v2:172.21.15.159:6800/2966947684 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f51b8103f20 con 0x7f51a00391b0 2024-09-19T07:23:09.231 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.590+0000 7f51bea00640 1 -- 172.21.15.159:0/643899917 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f51a00391b0 msgr2=0x7f51a003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:09.231 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.590+0000 7f51bea00640 1 --2- 172.21.15.159:0/643899917 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f51a00391b0 0x7f51a003b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f51a40098c0 tx=0x7f51a4006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:09.231 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.590+0000 7f51bea00640 1 -- 172.21.15.159:0/643899917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b807de80 msgr2=0x7f51b819d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:09.231 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.590+0000 7f51bea00640 1 --2- 172.21.15.159:0/643899917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b807de80 0x7f51b819d6b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0097f0 tx=0x7f51ac030ec0 comp rx=0 tx=0).stop 2024-09-19T07:23:09.231 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.590+0000 7f51bea00640 1 -- 172.21.15.159:0/643899917 shutdown_connections 2024-09-19T07:23:09.231 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.590+0000 7f51bea00640 1 --2- 172.21.15.159:0/643899917 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f51a00391b0 0x7f51a003b670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:09.231 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.590+0000 7f51bea00640 1 --2- 172.21.15.159:0/643899917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51b807de80 0x7f51b819d6b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:09.231 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.590+0000 7f51bea00640 1 -- 172.21.15.159:0/643899917 >> 172.21.15.159:0/643899917 conn(0x7f51b8102c10 msgr2=0x7f51b8103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:09.231 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.591+0000 7f51bea00640 1 -- 172.21.15.159:0/643899917 shutdown_connections 2024-09-19T07:23:09.231 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:08.591+0000 7f51bea00640 1 -- 172.21.15.159:0/643899917 wait complete. 2024-09-19T07:23:09.761 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:09 smithi159 ceph-mon[30472]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:23:09.761 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:09 smithi159 ceph-mon[30472]: Generating ssh key... 2024-09-19T07:23:09.761 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:09 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' 2024-09-19T07:23:09.761 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:09 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' 2024-09-19T07:23:10.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD40dGyPTYYo29uJxWEgmfUplFmtY/xf37wV6O9MUpHiatxI+Xq4WCYJxQT9Nf7jB0ijS/qxKZeCG7xng6vAsSAEnxwSLxTPNOYI0bkfLZvhpplD5dOg/tlrycCkXsotbfeosP2k6atNnCIlQV5E+FSQTEuqMfV4IoFerFyXULbJUP7rjPfIyxCmNCPKzlTipF6AHCvy2CI1iu29SRz4WYD+Cmlo/3bXmXdid0cPHPHsgjg4WNorQE3fBfxMUZjTemcjPVxElNRrSVI8xp4NIG7mx2Gy1B8J2meyGLSTtR/qLwHWB56lhKM/+9ZIjdYR3AA1U7jolLD/4CEEYCBG6IRrNGo74rRuEHTCLxK3Z8vPfNEy1C915LPt9vw0zNl4Z08N7zNy82vrvaScR6w9lornEKd9jrDnRmXfafxsOinDNZOf53qAa2mU1tvcZM4kgQbAG31WAIdl539kvmOlcwXaaORGdVn7eKVPC2lg1+0gJYlcNhXy9JcW4meMAO0/U0= ceph-c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:23:10.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.765+0000 7fd747e00640 1 Processor -- start 2024-09-19T07:23:10.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.765+0000 7fd747e00640 1 -- start start 2024-09-19T07:23:10.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.766+0000 7fd747e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd74007c340 0x7fd74007c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:10.656 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.766+0000 7fd747e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd74007e070 con 0x7fd74007c340 2024-09-19T07:23:10.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.766+0000 7fd745400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd74007c340 0x7fd74007c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:10.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.766+0000 7fd745400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd74007c340 0x7fd74007c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49858/0 (socket says 172.21.15.159:49858) 2024-09-19T07:23:10.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.766+0000 7fd745400640 1 -- 172.21.15.159:0/2038822103 learned_addr learned my addr 172.21.15.159:0/2038822103 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:10.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.766+0000 7fd745400640 1 -- 172.21.15.159:0/2038822103 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74007cc80 con 0x7fd74007c340 2024-09-19T07:23:10.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.767+0000 7fd745400640 1 --2- 172.21.15.159:0/2038822103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd74007c340 0x7fd74007c740 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd734009820 tx=0x7fd73402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50053409cb258ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:10.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.768+0000 7fd73fe00640 1 -- 172.21.15.159:0/2038822103 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd734037070 con 0x7fd74007c340 2024-09-19T07:23:10.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.768+0000 7fd73fe00640 1 -- 172.21.15.159:0/2038822103 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd734032900 con 0x7fd74007c340 2024-09-19T07:23:10.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.768+0000 7fd73fe00640 1 -- 172.21.15.159:0/2038822103 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd734030850 con 0x7fd74007c340 2024-09-19T07:23:10.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.768+0000 7fd747e00640 1 -- 172.21.15.159:0/2038822103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd74007c340 msgr2=0x7fd74007c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:10.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.768+0000 7fd747e00640 1 --2- 172.21.15.159:0/2038822103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd74007c340 0x7fd74007c740 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd734009820 tx=0x7fd73402a270 comp rx=0 tx=0).stop 2024-09-19T07:23:10.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.768+0000 7fd747e00640 1 -- 172.21.15.159:0/2038822103 shutdown_connections 2024-09-19T07:23:10.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.768+0000 7fd747e00640 1 --2- 172.21.15.159:0/2038822103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd74007c340 0x7fd74007c740 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:10.657 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.768+0000 7fd747e00640 1 -- 172.21.15.159:0/2038822103 >> 172.21.15.159:0/2038822103 conn(0x7fd740102dc0 msgr2=0x7fd7401051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:10.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.769+0000 7fd747e00640 1 -- 172.21.15.159:0/2038822103 shutdown_connections 2024-09-19T07:23:10.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.769+0000 7fd747e00640 1 -- 172.21.15.159:0/2038822103 wait complete. 2024-09-19T07:23:10.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.769+0000 7fd747e00640 1 Processor -- start 2024-09-19T07:23:10.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.770+0000 7fd747e00640 1 -- start start 2024-09-19T07:23:10.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.770+0000 7fd747e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd74007c340 0x7fd7401a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:10.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.770+0000 7fd747e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd740109950 con 0x7fd74007c340 2024-09-19T07:23:10.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.770+0000 7fd745400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd74007c340 0x7fd7401a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:10.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.770+0000 7fd745400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd74007c340 0x7fd7401a1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49872/0 (socket says 172.21.15.159:49872) 2024-09-19T07:23:10.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.770+0000 7fd745400640 1 -- 172.21.15.159:0/3225602661 learned_addr learned my addr 172.21.15.159:0/3225602661 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:10.658 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.770+0000 7fd745400640 1 -- 172.21.15.159:0/3225602661 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7401a2280 con 0x7fd74007c340 2024-09-19T07:23:10.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.771+0000 7fd745400640 1 --2- 172.21.15.159:0/3225602661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd74007c340 0x7fd7401a1d40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd734032ce0 tx=0x7fd734032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:10.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.771+0000 7fd73e000640 1 -- 172.21.15.159:0/3225602661 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd734042070 con 0x7fd74007c340 2024-09-19T07:23:10.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.771+0000 7fd73e000640 1 -- 172.21.15.159:0/3225602661 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd7340308c0 con 0x7fd74007c340 2024-09-19T07:23:10.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.771+0000 7fd747e00640 1 -- 172.21.15.159:0/3225602661 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7401a2510 con 0x7fd74007c340 2024-09-19T07:23:10.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.771+0000 7fd73e000640 1 -- 172.21.15.159:0/3225602661 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd734037040 con 0x7fd74007c340 2024-09-19T07:23:10.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.771+0000 7fd747e00640 1 -- 172.21.15.159:0/3225602661 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7401a5200 con 0x7fd74007c340 2024-09-19T07:23:10.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.772+0000 7fd73e000640 1 -- 172.21.15.159:0/3225602661 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fd734045070 con 0x7fd74007c340 2024-09-19T07:23:10.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.772+0000 7fd747e00640 1 -- 172.21.15.159:0/3225602661 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd74007c7c0 con 0x7fd74007c340 2024-09-19T07:23:10.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.773+0000 7fd73e000640 1 --2- 172.21.15.159:0/3225602661 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fd728039200 0x7fd72803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:10.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.773+0000 7fd73e000640 1 -- 172.21.15.159:0/3225602661 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd73406c3b0 con 0x7fd74007c340 2024-09-19T07:23:10.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.773+0000 7fd744a00640 1 --2- 172.21.15.159:0/3225602661 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fd728039200 0x7fd72803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:10.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.773+0000 7fd744a00640 1 --2- 172.21.15.159:0/3225602661 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fd728039200 0x7fd72803b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0098c0 tx=0x7fd72c006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:10.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.777+0000 7fd73e000640 1 -- 172.21.15.159:0/3225602661 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd73404b170 con 0x7fd74007c340 2024-09-19T07:23:10.659 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.954+0000 7fd747e00640 1 -- 172.21.15.159:0/3225602661 --> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fd74002d9c0 con 0x7fd728039200 2024-09-19T07:23:10.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.955+0000 7fd73e000640 1 -- 172.21.15.159:0/3225602661 <== mgr.14116 v2:172.21.15.159:6800/2966947684 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fd74002d9c0 con 0x7fd728039200 2024-09-19T07:23:10.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.957+0000 7fd747e00640 1 -- 172.21.15.159:0/3225602661 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fd728039200 msgr2=0x7fd72803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:10.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.957+0000 7fd747e00640 1 --2- 172.21.15.159:0/3225602661 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fd728039200 0x7fd72803b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0098c0 tx=0x7fd72c006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:10.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.957+0000 7fd747e00640 1 -- 172.21.15.159:0/3225602661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd74007c340 msgr2=0x7fd7401a1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:10.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.957+0000 7fd747e00640 1 --2- 172.21.15.159:0/3225602661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd74007c340 0x7fd7401a1d40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd734032ce0 tx=0x7fd734032d10 comp rx=0 tx=0).stop 2024-09-19T07:23:10.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.957+0000 7fd747e00640 1 -- 172.21.15.159:0/3225602661 shutdown_connections 2024-09-19T07:23:10.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.957+0000 7fd747e00640 1 --2- 172.21.15.159:0/3225602661 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fd728039200 0x7fd72803b6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:10.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.957+0000 7fd747e00640 1 --2- 172.21.15.159:0/3225602661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd74007c340 0x7fd7401a1d40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:10.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.957+0000 7fd747e00640 1 -- 172.21.15.159:0/3225602661 >> 172.21.15.159:0/3225602661 conn(0x7fd740102dc0 msgr2=0x7fd740198900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:10.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.958+0000 7fd747e00640 1 -- 172.21.15.159:0/3225602661 shutdown_connections 2024-09-19T07:23:10.660 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:09.958+0000 7fd747e00640 1 -- 172.21.15.159:0/3225602661 wait complete. 2024-09-19T07:23:10.660 INFO:teuthology.orchestra.run.smithi159.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T07:23:10.660 INFO:teuthology.orchestra.run.smithi159.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T07:23:10.667 INFO:teuthology.orchestra.run.smithi159.stdout:Adding host smithi159... 2024-09-19T07:23:10.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:10 smithi159 ceph-mon[30472]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:23:12.596 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:12 smithi159 ceph-mon[30472]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi159", "addr": "172.21.15.159", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:23:13.478 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:13 smithi159 ceph-mon[30472]: Deploying cephadm binary to smithi159 2024-09-19T07:23:15.833 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout Added host 'smithi159' with addr '172.21.15.159' 2024-09-19T07:23:15.833 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.176+0000 7fb405800640 1 Processor -- start 2024-09-19T07:23:15.833 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.176+0000 7fb405800640 1 -- start start 2024-09-19T07:23:15.833 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.177+0000 7fb405800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb400107680 0x7fb400107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:15.833 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.177+0000 7fb405800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb400107fc0 con 0x7fb400107680 2024-09-19T07:23:15.833 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.177+0000 7fb3fea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb400107680 0x7fb400107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:15.833 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.177+0000 7fb3fea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb400107680 0x7fb400107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49882/0 (socket says 172.21.15.159:49882) 2024-09-19T07:23:15.834 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.177+0000 7fb3fea00640 1 -- 172.21.15.159:0/1271029606 learned_addr learned my addr 172.21.15.159:0/1271029606 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:15.834 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.177+0000 7fb3fea00640 1 -- 172.21.15.159:0/1271029606 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb400108140 con 0x7fb400107680 2024-09-19T07:23:15.834 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.178+0000 7fb3fea00640 1 --2- 172.21.15.159:0/1271029606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb400107680 0x7fb400107a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009820 tx=0x7fb3f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cf6b8e0a007e08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:15.834 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.178+0000 7fb3fd600640 1 -- 172.21.15.159:0/1271029606 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3f0037070 con 0x7fb400107680 2024-09-19T07:23:15.834 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.179+0000 7fb3fd600640 1 -- 172.21.15.159:0/1271029606 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb3f0032900 con 0x7fb400107680 2024-09-19T07:23:15.834 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.179+0000 7fb3fd600640 1 -- 172.21.15.159:0/1271029606 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3f0030850 con 0x7fb400107680 2024-09-19T07:23:15.834 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.179+0000 7fb405800640 1 -- 172.21.15.159:0/1271029606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb400107680 msgr2=0x7fb400107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:15.834 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.179+0000 7fb405800640 1 --2- 172.21.15.159:0/1271029606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb400107680 0x7fb400107a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009820 tx=0x7fb3f002a270 comp rx=0 tx=0).stop 2024-09-19T07:23:15.834 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.179+0000 7fb405800640 1 -- 172.21.15.159:0/1271029606 shutdown_connections 2024-09-19T07:23:15.834 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.179+0000 7fb405800640 1 --2- 172.21.15.159:0/1271029606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb400107680 0x7fb400107a80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:15.834 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.179+0000 7fb405800640 1 -- 172.21.15.159:0/1271029606 >> 172.21.15.159:0/1271029606 conn(0x7fb400102eb0 msgr2=0x7fb4001052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:15.834 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.179+0000 7fb405800640 1 -- 172.21.15.159:0/1271029606 shutdown_connections 2024-09-19T07:23:15.834 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.179+0000 7fb405800640 1 -- 172.21.15.159:0/1271029606 wait complete. 2024-09-19T07:23:15.835 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.180+0000 7fb405800640 1 Processor -- start 2024-09-19T07:23:15.835 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.180+0000 7fb405800640 1 -- start start 2024-09-19T07:23:15.835 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.181+0000 7fb405800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb400107680 0x7fb400194600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:15.835 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.181+0000 7fb405800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb40010baf0 con 0x7fb400107680 2024-09-19T07:23:15.835 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.181+0000 7fb3fea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb400107680 0x7fb400194600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:15.835 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.181+0000 7fb3fea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb400107680 0x7fb400194600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49892/0 (socket says 172.21.15.159:49892) 2024-09-19T07:23:15.835 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.181+0000 7fb3fea00640 1 -- 172.21.15.159:0/856752738 learned_addr learned my addr 172.21.15.159:0/856752738 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:15.835 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.181+0000 7fb3fea00640 1 -- 172.21.15.159:0/856752738 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb400194b40 con 0x7fb400107680 2024-09-19T07:23:15.835 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.181+0000 7fb3fea00640 1 --2- 172.21.15.159:0/856752738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb400107680 0x7fb400194600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00097f0 tx=0x7fb3f0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:15.835 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.182+0000 7fb3f7400640 1 -- 172.21.15.159:0/856752738 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3f0042070 con 0x7fb400107680 2024-09-19T07:23:15.835 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.182+0000 7fb3f7400640 1 -- 172.21.15.159:0/856752738 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb3f0032ca0 con 0x7fb400107680 2024-09-19T07:23:15.835 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.182+0000 7fb3f7400640 1 -- 172.21.15.159:0/856752738 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3f0037040 con 0x7fb400107680 2024-09-19T07:23:15.835 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.182+0000 7fb405800640 1 -- 172.21.15.159:0/856752738 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb400194dd0 con 0x7fb400107680 2024-09-19T07:23:15.836 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.182+0000 7fb405800640 1 -- 172.21.15.159:0/856752738 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb400191390 con 0x7fb400107680 2024-09-19T07:23:15.836 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.183+0000 7fb3f7400640 1 -- 172.21.15.159:0/856752738 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fb3f0045070 con 0x7fb400107680 2024-09-19T07:23:15.836 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.183+0000 7fb405800640 1 -- 172.21.15.159:0/856752738 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb400194fa0 con 0x7fb400107680 2024-09-19T07:23:15.836 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.183+0000 7fb3f7400640 1 --2- 172.21.15.159:0/856752738 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fb3e4039200 0x7fb3e403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:15.836 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.184+0000 7fb3f7400640 1 -- 172.21.15.159:0/856752738 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb3f006c2d0 con 0x7fb400107680 2024-09-19T07:23:15.836 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.184+0000 7fb3fe000640 1 --2- 172.21.15.159:0/856752738 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fb3e4039200 0x7fb3e403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:15.836 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.184+0000 7fb3fe000640 1 --2- 172.21.15.159:0/856752738 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fb3e4039200 0x7fb3e403b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec0098c0 tx=0x7fb3ec006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:15.836 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.188+0000 7fb3f7400640 1 -- 172.21.15.159:0/856752738 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb400194fa0 con 0x7fb400107680 2024-09-19T07:23:15.836 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:11.372+0000 7fb405800640 1 -- 172.21.15.159:0/856752738 --> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi159", "addr": "172.21.15.159", "target": ["mon-mgr", ""]}) -- 0x7fb400191b30 con 0x7fb3e4039200 2024-09-19T07:23:15.836 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:15.168+0000 7fb3f7400640 1 -- 172.21.15.159:0/856752738 <== mgr.14116 v2:172.21.15.159:6800/2966947684 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb400191b30 con 0x7fb3e4039200 2024-09-19T07:23:15.837 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:15.172+0000 7fb405800640 1 -- 172.21.15.159:0/856752738 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fb3e4039200 msgr2=0x7fb3e403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:15.837 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:15.172+0000 7fb405800640 1 --2- 172.21.15.159:0/856752738 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fb3e4039200 0x7fb3e403b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec0098c0 tx=0x7fb3ec006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:15.837 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:15.172+0000 7fb405800640 1 -- 172.21.15.159:0/856752738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb400107680 msgr2=0x7fb400194600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:15.837 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:15.172+0000 7fb405800640 1 --2- 172.21.15.159:0/856752738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb400107680 0x7fb400194600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00097f0 tx=0x7fb3f0030ec0 comp rx=0 tx=0).stop 2024-09-19T07:23:15.837 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:15.172+0000 7fb405800640 1 -- 172.21.15.159:0/856752738 shutdown_connections 2024-09-19T07:23:15.837 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:15.172+0000 7fb405800640 1 --2- 172.21.15.159:0/856752738 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fb3e4039200 0x7fb3e403b6c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:15.837 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:15.172+0000 7fb405800640 1 --2- 172.21.15.159:0/856752738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb400107680 0x7fb400194600 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:15.837 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:15.172+0000 7fb405800640 1 -- 172.21.15.159:0/856752738 >> 172.21.15.159:0/856752738 conn(0x7fb400102eb0 msgr2=0x7fb4001039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:15.837 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:15.173+0000 7fb405800640 1 -- 172.21.15.159:0/856752738 shutdown_connections 2024-09-19T07:23:15.837 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:15.173+0000 7fb405800640 1 -- 172.21.15.159:0/856752738 wait complete. 2024-09-19T07:23:15.838 INFO:teuthology.orchestra.run.smithi159.stdout:Deploying unmanaged mon service... 2024-09-19T07:23:16.434 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:16 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' 2024-09-19T07:23:16.434 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:16 smithi159 ceph-mon[30472]: Added host smithi159 2024-09-19T07:23:16.434 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:16 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:23:17.756 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T07:23:17.757 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.433+0000 7ff365400640 1 Processor -- start 2024-09-19T07:23:17.757 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.433+0000 7ff365400640 1 -- start start 2024-09-19T07:23:17.757 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.433+0000 7ff365400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff360107440 0x7ff360107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:17.757 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.433+0000 7ff365400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff360107e10 con 0x7ff360107440 2024-09-19T07:23:17.757 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.433+0000 7ff35fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff360107440 0x7ff360107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:17.757 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.434+0000 7ff35fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff360107440 0x7ff360107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59880/0 (socket says 172.21.15.159:59880) 2024-09-19T07:23:17.757 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.434+0000 7ff35fe00640 1 -- 172.21.15.159:0/3267318657 learned_addr learned my addr 172.21.15.159:0/3267318657 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:17.757 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.434+0000 7ff35fe00640 1 -- 172.21.15.159:0/3267318657 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff360108640 con 0x7ff360107440 2024-09-19T07:23:17.757 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.435+0000 7ff35fe00640 1 --2- 172.21.15.159:0/3267318657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff360107440 0x7ff360107840 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff350009e80 tx=0x7ff35002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5116e0bc259e841e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:17.757 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.435+0000 7ff35f400640 1 -- 172.21.15.159:0/3267318657 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff350036070 con 0x7ff360107440 2024-09-19T07:23:17.757 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.435+0000 7ff35f400640 1 -- 172.21.15.159:0/3267318657 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff3500318c0 con 0x7ff360107440 2024-09-19T07:23:17.757 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.435+0000 7ff35f400640 1 -- 172.21.15.159:0/3267318657 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff350007ab0 con 0x7ff360107440 2024-09-19T07:23:17.758 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.436+0000 7ff365400640 1 -- 172.21.15.159:0/3267318657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff360107440 msgr2=0x7ff360107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:17.758 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.436+0000 7ff365400640 1 --2- 172.21.15.159:0/3267318657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff360107440 0x7ff360107840 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff350009e80 tx=0x7ff35002a7a0 comp rx=0 tx=0).stop 2024-09-19T07:23:17.758 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.436+0000 7ff365400640 1 -- 172.21.15.159:0/3267318657 shutdown_connections 2024-09-19T07:23:17.758 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.436+0000 7ff365400640 1 --2- 172.21.15.159:0/3267318657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff360107440 0x7ff360107840 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:17.758 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.436+0000 7ff365400640 1 -- 172.21.15.159:0/3267318657 >> 172.21.15.159:0/3267318657 conn(0x7ff360102c10 msgr2=0x7ff360105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:17.758 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.436+0000 7ff365400640 1 -- 172.21.15.159:0/3267318657 shutdown_connections 2024-09-19T07:23:17.758 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.436+0000 7ff365400640 1 -- 172.21.15.159:0/3267318657 wait complete. 2024-09-19T07:23:17.758 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.437+0000 7ff365400640 1 Processor -- start 2024-09-19T07:23:17.758 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.437+0000 7ff365400640 1 -- start start 2024-09-19T07:23:17.758 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.437+0000 7ff365400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff360107440 0x7ff36019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:17.758 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.437+0000 7ff365400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff36010b9f0 con 0x7ff360107440 2024-09-19T07:23:17.758 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.437+0000 7ff35fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff360107440 0x7ff36019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:17.758 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.437+0000 7ff35fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff360107440 0x7ff36019d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59888/0 (socket says 172.21.15.159:59888) 2024-09-19T07:23:17.758 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.437+0000 7ff35fe00640 1 -- 172.21.15.159:0/3251090457 learned_addr learned my addr 172.21.15.159:0/3251090457 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:17.759 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.438+0000 7ff35fe00640 1 -- 172.21.15.159:0/3251090457 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff36019dc50 con 0x7ff360107440 2024-09-19T07:23:17.759 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.438+0000 7ff35fe00640 1 --2- 172.21.15.159:0/3251090457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff360107440 0x7ff36019d710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff35002acc0 tx=0x7ff350031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:17.759 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.438+0000 7ff35d600640 1 -- 172.21.15.159:0/3251090457 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff350036050 con 0x7ff360107440 2024-09-19T07:23:17.759 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.438+0000 7ff365400640 1 -- 172.21.15.159:0/3251090457 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff36019dee0 con 0x7ff360107440 2024-09-19T07:23:17.759 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.438+0000 7ff35d600640 1 -- 172.21.15.159:0/3251090457 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff350037040 con 0x7ff360107440 2024-09-19T07:23:17.759 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.438+0000 7ff35d600640 1 -- 172.21.15.159:0/3251090457 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff35003d890 con 0x7ff360107440 2024-09-19T07:23:17.759 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.438+0000 7ff365400640 1 -- 172.21.15.159:0/3251090457 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3601a0bd0 con 0x7ff360107440 2024-09-19T07:23:17.759 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.439+0000 7ff35d600640 1 -- 172.21.15.159:0/3251090457 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7ff350044020 con 0x7ff360107440 2024-09-19T07:23:17.759 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.440+0000 7ff365400640 1 -- 172.21.15.159:0/3251090457 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff36010bb70 con 0x7ff360107440 2024-09-19T07:23:17.760 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.440+0000 7ff35d600640 1 --2- 172.21.15.159:0/3251090457 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7ff3440391b0 0x7ff34403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:17.760 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.440+0000 7ff35d600640 1 -- 172.21.15.159:0/3251090457 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff350042020 con 0x7ff360107440 2024-09-19T07:23:17.760 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.440+0000 7ff357400640 1 --2- 172.21.15.159:0/3251090457 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7ff3440391b0 0x7ff34403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:17.760 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.441+0000 7ff357400640 1 --2- 172.21.15.159:0/3251090457 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7ff3440391b0 0x7ff34403b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff3480098c0 tx=0x7ff348006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:17.760 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.445+0000 7ff35d600640 1 -- 172.21.15.159:0/3251090457 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff35004b020 con 0x7ff360107440 2024-09-19T07:23:17.760 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.642+0000 7ff365400640 1 -- 172.21.15.159:0/3251090457 --> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff360103eb0 con 0x7ff3440391b0 2024-09-19T07:23:17.760 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.648+0000 7ff35d600640 1 -- 172.21.15.159:0/3251090457 <== mgr.14116 v2:172.21.15.159:6800/2966947684 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff360103eb0 con 0x7ff3440391b0 2024-09-19T07:23:17.760 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.650+0000 7ff365400640 1 -- 172.21.15.159:0/3251090457 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7ff3440391b0 msgr2=0x7ff34403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:17.760 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.650+0000 7ff365400640 1 --2- 172.21.15.159:0/3251090457 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7ff3440391b0 0x7ff34403b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff3480098c0 tx=0x7ff348006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:17.761 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.650+0000 7ff365400640 1 -- 172.21.15.159:0/3251090457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff360107440 msgr2=0x7ff36019d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:17.761 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.650+0000 7ff365400640 1 --2- 172.21.15.159:0/3251090457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff360107440 0x7ff36019d710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff35002acc0 tx=0x7ff350031ca0 comp rx=0 tx=0).stop 2024-09-19T07:23:17.761 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.650+0000 7ff365400640 1 -- 172.21.15.159:0/3251090457 shutdown_connections 2024-09-19T07:23:17.761 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.650+0000 7ff365400640 1 --2- 172.21.15.159:0/3251090457 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7ff3440391b0 0x7ff34403b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:17.761 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.650+0000 7ff365400640 1 --2- 172.21.15.159:0/3251090457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff360107440 0x7ff36019d710 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:17.761 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.650+0000 7ff365400640 1 -- 172.21.15.159:0/3251090457 >> 172.21.15.159:0/3251090457 conn(0x7ff360102c10 msgr2=0x7ff3601037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:17.761 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.650+0000 7ff365400640 1 -- 172.21.15.159:0/3251090457 shutdown_connections 2024-09-19T07:23:17.761 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:16.650+0000 7ff365400640 1 -- 172.21.15.159:0/3251090457 wait complete. 2024-09-19T07:23:17.761 INFO:teuthology.orchestra.run.smithi159.stdout:Deploying unmanaged mgr service... 2024-09-19T07:23:18.014 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:17 smithi159 ceph-mon[30472]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:23:18.014 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:17 smithi159 ceph-mon[30472]: Saving service mon spec with placement count:5 2024-09-19T07:23:18.014 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:17 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' 2024-09-19T07:23:19.474 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T07:23:19.474 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.406+0000 7ff1dea00640 1 Processor -- start 2024-09-19T07:23:19.474 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.406+0000 7ff1dea00640 1 -- start start 2024-09-19T07:23:19.474 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.407+0000 7ff1dea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1d8109870 0x7ff1d8109c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:19.474 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.407+0000 7ff1dea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d810a240 con 0x7ff1d8109870 2024-09-19T07:23:19.474 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.407+0000 7ff1d7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1d8109870 0x7ff1d8109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:19.474 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.407+0000 7ff1d7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1d8109870 0x7ff1d8109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59904/0 (socket says 172.21.15.159:59904) 2024-09-19T07:23:19.474 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.407+0000 7ff1d7e00640 1 -- 172.21.15.159:0/522806588 learned_addr learned my addr 172.21.15.159:0/522806588 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:19.474 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.408+0000 7ff1d7e00640 1 -- 172.21.15.159:0/522806588 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d810aa10 con 0x7ff1d8109870 2024-09-19T07:23:19.474 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.408+0000 7ff1d7e00640 1 --2- 172.21.15.159:0/522806588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1d8109870 0x7ff1d8109c70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8009820 tx=0x7ff1c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196d8d8a69192545 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:19.474 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.409+0000 7ff1d7400640 1 -- 172.21.15.159:0/522806588 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1c8037070 con 0x7ff1d8109870 2024-09-19T07:23:19.474 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.409+0000 7ff1d7400640 1 -- 172.21.15.159:0/522806588 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff1c8032900 con 0x7ff1d8109870 2024-09-19T07:23:19.475 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.409+0000 7ff1d7400640 1 -- 172.21.15.159:0/522806588 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1c8030850 con 0x7ff1d8109870 2024-09-19T07:23:19.475 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.409+0000 7ff1dea00640 1 -- 172.21.15.159:0/522806588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1d8109870 msgr2=0x7ff1d8109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:19.475 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.409+0000 7ff1dea00640 1 --2- 172.21.15.159:0/522806588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1d8109870 0x7ff1d8109c70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8009820 tx=0x7ff1c802a270 comp rx=0 tx=0).stop 2024-09-19T07:23:19.475 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.409+0000 7ff1dea00640 1 -- 172.21.15.159:0/522806588 shutdown_connections 2024-09-19T07:23:19.475 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.409+0000 7ff1dea00640 1 --2- 172.21.15.159:0/522806588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1d8109870 0x7ff1d8109c70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:19.475 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.409+0000 7ff1dea00640 1 -- 172.21.15.159:0/522806588 >> 172.21.15.159:0/522806588 conn(0x7ff1d807cf40 msgr2=0x7ff1d807d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:19.475 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.410+0000 7ff1dea00640 1 -- 172.21.15.159:0/522806588 shutdown_connections 2024-09-19T07:23:19.475 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.410+0000 7ff1dea00640 1 -- 172.21.15.159:0/522806588 wait complete. 2024-09-19T07:23:19.475 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.410+0000 7ff1dea00640 1 Processor -- start 2024-09-19T07:23:19.475 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.411+0000 7ff1dea00640 1 -- start start 2024-09-19T07:23:19.475 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.411+0000 7ff1dea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1d8109870 0x7ff1d819d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:19.475 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.411+0000 7ff1dea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d810ddc0 con 0x7ff1d8109870 2024-09-19T07:23:19.475 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.411+0000 7ff1d7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1d8109870 0x7ff1d819d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:19.476 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.411+0000 7ff1d7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1d8109870 0x7ff1d819d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59906/0 (socket says 172.21.15.159:59906) 2024-09-19T07:23:19.476 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.411+0000 7ff1d7e00640 1 -- 172.21.15.159:0/3898483347 learned_addr learned my addr 172.21.15.159:0/3898483347 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:19.476 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.412+0000 7ff1d7e00640 1 -- 172.21.15.159:0/3898483347 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d819dd20 con 0x7ff1d8109870 2024-09-19T07:23:19.476 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.412+0000 7ff1d7e00640 1 --2- 172.21.15.159:0/3898483347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1d8109870 0x7ff1d819d7e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8032ce0 tx=0x7ff1c8032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:19.476 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.412+0000 7ff1d5600640 1 -- 172.21.15.159:0/3898483347 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1c8042070 con 0x7ff1d8109870 2024-09-19T07:23:19.476 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.412+0000 7ff1d5600640 1 -- 172.21.15.159:0/3898483347 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff1c80308c0 con 0x7ff1d8109870 2024-09-19T07:23:19.476 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.412+0000 7ff1d5600640 1 -- 172.21.15.159:0/3898483347 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1c8037040 con 0x7ff1d8109870 2024-09-19T07:23:19.476 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.412+0000 7ff1dea00640 1 -- 172.21.15.159:0/3898483347 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d819dfb0 con 0x7ff1d8109870 2024-09-19T07:23:19.476 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.412+0000 7ff1dea00640 1 -- 172.21.15.159:0/3898483347 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d81a0ca0 con 0x7ff1d8109870 2024-09-19T07:23:19.476 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.413+0000 7ff1dea00640 1 -- 172.21.15.159:0/3898483347 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d8109c70 con 0x7ff1d8109870 2024-09-19T07:23:19.476 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.413+0000 7ff1d5600640 1 -- 172.21.15.159:0/3898483347 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7ff1c8045070 con 0x7ff1d8109870 2024-09-19T07:23:19.476 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.414+0000 7ff1d5600640 1 --2- 172.21.15.159:0/3898483347 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7ff1bc039200 0x7ff1bc03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:19.476 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.414+0000 7ff1d5600640 1 -- 172.21.15.159:0/3898483347 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff1c806c300 con 0x7ff1d8109870 2024-09-19T07:23:19.477 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.414+0000 7ff1cf400640 1 --2- 172.21.15.159:0/3898483347 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7ff1bc039200 0x7ff1bc03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:19.477 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.415+0000 7ff1cf400640 1 --2- 172.21.15.159:0/3898483347 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7ff1bc039200 0x7ff1bc03b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00098c0 tx=0x7ff1c0006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:19.477 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.419+0000 7ff1d5600640 1 -- 172.21.15.159:0/3898483347 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff1c8045360 con 0x7ff1d8109870 2024-09-19T07:23:19.477 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.628+0000 7ff1dea00640 1 -- 172.21.15.159:0/3898483347 --> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff1d8107b90 con 0x7ff1bc039200 2024-09-19T07:23:19.477 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.633+0000 7ff1d5600640 1 -- 172.21.15.159:0/3898483347 <== mgr.14116 v2:172.21.15.159:6800/2966947684 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff1d8107b90 con 0x7ff1bc039200 2024-09-19T07:23:19.477 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.635+0000 7ff1dea00640 1 -- 172.21.15.159:0/3898483347 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7ff1bc039200 msgr2=0x7ff1bc03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:19.477 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.635+0000 7ff1dea00640 1 --2- 172.21.15.159:0/3898483347 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7ff1bc039200 0x7ff1bc03b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00098c0 tx=0x7ff1c0006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:19.477 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.635+0000 7ff1dea00640 1 -- 172.21.15.159:0/3898483347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1d8109870 msgr2=0x7ff1d819d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:19.477 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.635+0000 7ff1dea00640 1 --2- 172.21.15.159:0/3898483347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1d8109870 0x7ff1d819d7e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8032ce0 tx=0x7ff1c8032d10 comp rx=0 tx=0).stop 2024-09-19T07:23:19.477 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.636+0000 7ff1dea00640 1 -- 172.21.15.159:0/3898483347 shutdown_connections 2024-09-19T07:23:19.477 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.636+0000 7ff1dea00640 1 --2- 172.21.15.159:0/3898483347 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7ff1bc039200 0x7ff1bc03b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:19.477 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.636+0000 7ff1dea00640 1 --2- 172.21.15.159:0/3898483347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1d8109870 0x7ff1d819d7e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:19.477 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.636+0000 7ff1dea00640 1 -- 172.21.15.159:0/3898483347 >> 172.21.15.159:0/3898483347 conn(0x7ff1d807cf40 msgr2=0x7ff1d8107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:19.478 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.636+0000 7ff1dea00640 1 -- 172.21.15.159:0/3898483347 shutdown_connections 2024-09-19T07:23:19.478 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:18.636+0000 7ff1dea00640 1 -- 172.21.15.159:0/3898483347 wait complete. 2024-09-19T07:23:19.809 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:19 smithi159 ceph-mon[30472]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:23:19.809 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:19 smithi159 ceph-mon[30472]: Saving service mgr spec with placement count:2 2024-09-19T07:23:19.809 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:19 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' 2024-09-19T07:23:19.809 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:19 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' 2024-09-19T07:23:19.809 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:19 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' 2024-09-19T07:23:21.234 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.062+0000 7f0efec00640 1 Processor -- start 2024-09-19T07:23:21.239 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.063+0000 7f0efec00640 1 -- start start 2024-09-19T07:23:21.239 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.063+0000 7f0efec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef8107680 0x7f0ef8107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:21.239 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.063+0000 7f0efec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef8108050 con 0x7f0ef8107680 2024-09-19T07:23:21.239 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.063+0000 7f0ef7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef8107680 0x7f0ef8107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:21.239 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.063+0000 7f0ef7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef8107680 0x7f0ef8107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59920/0 (socket says 172.21.15.159:59920) 2024-09-19T07:23:21.239 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.063+0000 7f0ef7e00640 1 -- 172.21.15.159:0/2488894878 learned_addr learned my addr 172.21.15.159:0/2488894878 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:21.239 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.064+0000 7f0ef7e00640 1 -- 172.21.15.159:0/2488894878 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef8108830 con 0x7f0ef8107680 2024-09-19T07:23:21.239 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.065+0000 7f0ef7e00640 1 --2- 172.21.15.159:0/2488894878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef8107680 0x7f0ef8107a80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009820 tx=0x7f0ee802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b59b8bc317ed43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:21.239 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.065+0000 7f0ef6a00640 1 -- 172.21.15.159:0/2488894878 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ee8037070 con 0x7f0ef8107680 2024-09-19T07:23:21.240 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.065+0000 7f0ef6a00640 1 -- 172.21.15.159:0/2488894878 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee8032920 con 0x7f0ef8107680 2024-09-19T07:23:21.240 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.065+0000 7f0ef6a00640 1 -- 172.21.15.159:0/2488894878 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ee8030810 con 0x7f0ef8107680 2024-09-19T07:23:21.240 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.066+0000 7f0efec00640 1 -- 172.21.15.159:0/2488894878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef8107680 msgr2=0x7f0ef8107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:21.240 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.066+0000 7f0efec00640 1 --2- 172.21.15.159:0/2488894878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef8107680 0x7f0ef8107a80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009820 tx=0x7f0ee802a270 comp rx=0 tx=0).stop 2024-09-19T07:23:21.240 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.066+0000 7f0efec00640 1 -- 172.21.15.159:0/2488894878 shutdown_connections 2024-09-19T07:23:21.240 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.066+0000 7f0efec00640 1 --2- 172.21.15.159:0/2488894878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef8107680 0x7f0ef8107a80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:21.240 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.066+0000 7f0efec00640 1 -- 172.21.15.159:0/2488894878 >> 172.21.15.159:0/2488894878 conn(0x7f0ef8102e50 msgr2=0x7f0ef8105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:21.240 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.066+0000 7f0efec00640 1 -- 172.21.15.159:0/2488894878 shutdown_connections 2024-09-19T07:23:21.240 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.066+0000 7f0efec00640 1 -- 172.21.15.159:0/2488894878 wait complete. 2024-09-19T07:23:21.240 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.067+0000 7f0efec00640 1 Processor -- start 2024-09-19T07:23:21.240 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.067+0000 7f0efec00640 1 -- start start 2024-09-19T07:23:21.240 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.067+0000 7f0efec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef8107680 0x7f0ef819d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:21.240 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.067+0000 7f0efec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef810bbe0 con 0x7f0ef8107680 2024-09-19T07:23:21.241 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.067+0000 7f0ef7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef8107680 0x7f0ef819d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:21.241 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.067+0000 7f0ef7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef8107680 0x7f0ef819d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59924/0 (socket says 172.21.15.159:59924) 2024-09-19T07:23:21.241 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.067+0000 7f0ef7e00640 1 -- 172.21.15.159:0/1432642144 learned_addr learned my addr 172.21.15.159:0/1432642144 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:21.241 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.068+0000 7f0ef7e00640 1 -- 172.21.15.159:0/1432642144 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef819de10 con 0x7f0ef8107680 2024-09-19T07:23:21.241 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.068+0000 7f0ef7e00640 1 --2- 172.21.15.159:0/1432642144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef8107680 0x7f0ef819d8d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80097f0 tx=0x7f0ee80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:21.241 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.068+0000 7f0ef4c00640 1 -- 172.21.15.159:0/1432642144 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ee8037040 con 0x7f0ef8107680 2024-09-19T07:23:21.241 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.068+0000 7f0ef4c00640 1 -- 172.21.15.159:0/1432642144 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee8030c40 con 0x7f0ef8107680 2024-09-19T07:23:21.241 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.068+0000 7f0ef4c00640 1 -- 172.21.15.159:0/1432642144 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ee803a5b0 con 0x7f0ef8107680 2024-09-19T07:23:21.241 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.068+0000 7f0efec00640 1 -- 172.21.15.159:0/1432642144 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef819e0a0 con 0x7f0ef8107680 2024-09-19T07:23:21.241 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.068+0000 7f0efec00640 1 -- 172.21.15.159:0/1432642144 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef81a0d90 con 0x7f0ef8107680 2024-09-19T07:23:21.241 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.069+0000 7f0ef4c00640 1 -- 172.21.15.159:0/1432642144 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f0ee803a7a0 con 0x7f0ef8107680 2024-09-19T07:23:21.241 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.069+0000 7f0efec00640 1 -- 172.21.15.159:0/1432642144 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef810bd60 con 0x7f0ef8107680 2024-09-19T07:23:21.242 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.070+0000 7f0ef4c00640 1 --2- 172.21.15.159:0/1432642144 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f0edc039200 0x7f0edc03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:21.242 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.070+0000 7f0ef4c00640 1 -- 172.21.15.159:0/1432642144 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0ee806c740 con 0x7f0ef8107680 2024-09-19T07:23:21.242 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.070+0000 7f0ef7400640 1 --2- 172.21.15.159:0/1432642144 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f0edc039200 0x7f0edc03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:21.242 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.071+0000 7f0ef7400640 1 --2- 172.21.15.159:0/1432642144 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f0edc039200 0x7f0edc03b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40098c0 tx=0x7f0ee4006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:21.242 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.075+0000 7f0ef4c00640 1 -- 172.21.15.159:0/1432642144 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0ee80371e0 con 0x7f0ef8107680 2024-09-19T07:23:21.242 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.267+0000 7f0efec00640 1 -- 172.21.15.159:0/1432642144 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f0ef8107b00 con 0x7f0ef8107680 2024-09-19T07:23:21.242 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.270+0000 7f0ef4c00640 1 -- 172.21.15.159:0/1432642144 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f0ee806c090 con 0x7f0ef8107680 2024-09-19T07:23:21.242 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.272+0000 7f0efec00640 1 -- 172.21.15.159:0/1432642144 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f0edc039200 msgr2=0x7f0edc03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:21.242 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.272+0000 7f0efec00640 1 --2- 172.21.15.159:0/1432642144 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f0edc039200 0x7f0edc03b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40098c0 tx=0x7f0ee4006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:21.242 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.272+0000 7f0efec00640 1 -- 172.21.15.159:0/1432642144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef8107680 msgr2=0x7f0ef819d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:21.242 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.272+0000 7f0efec00640 1 --2- 172.21.15.159:0/1432642144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef8107680 0x7f0ef819d8d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80097f0 tx=0x7f0ee80327b0 comp rx=0 tx=0).stop 2024-09-19T07:23:21.242 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.273+0000 7f0efec00640 1 -- 172.21.15.159:0/1432642144 shutdown_connections 2024-09-19T07:23:21.242 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.273+0000 7f0efec00640 1 --2- 172.21.15.159:0/1432642144 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f0edc039200 0x7f0edc03b6c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:21.243 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.273+0000 7f0efec00640 1 --2- 172.21.15.159:0/1432642144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef8107680 0x7f0ef819d8d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:21.243 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.273+0000 7f0efec00640 1 -- 172.21.15.159:0/1432642144 >> 172.21.15.159:0/1432642144 conn(0x7f0ef8102e50 msgr2=0x7f0ef81039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:21.243 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.273+0000 7f0efec00640 1 -- 172.21.15.159:0/1432642144 shutdown_connections 2024-09-19T07:23:21.243 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:20.273+0000 7f0efec00640 1 -- 172.21.15.159:0/1432642144 wait complete. 2024-09-19T07:23:21.588 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:21 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1432642144' entity='client.admin' 2024-09-19T07:23:23.102 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.176+0000 7fa543a00640 1 Processor -- start 2024-09-19T07:23:23.102 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.176+0000 7fa543a00640 1 -- start start 2024-09-19T07:23:23.102 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.176+0000 7fa543a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa53c109870 0x7fa53c109c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:23.102 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.177+0000 7fa543a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c10a240 con 0x7fa53c109870 2024-09-19T07:23:23.102 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.177+0000 7fa541000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa53c109870 0x7fa53c109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:23.102 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.177+0000 7fa541000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa53c109870 0x7fa53c109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49356/0 (socket says 172.21.15.159:49356) 2024-09-19T07:23:23.102 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.177+0000 7fa541000640 1 -- 172.21.15.159:0/1429878986 learned_addr learned my addr 172.21.15.159:0/1429878986 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:23.102 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.177+0000 7fa541000640 1 -- 172.21.15.159:0/1429878986 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53c10aa10 con 0x7fa53c109870 2024-09-19T07:23:23.102 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.178+0000 7fa541000640 1 --2- 172.21.15.159:0/1429878986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa53c109870 0x7fa53c109c70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa530009820 tx=0x7fa53002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc23b0042565fcfa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:23.102 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.179+0000 7fa53b400640 1 -- 172.21.15.159:0/1429878986 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa530037070 con 0x7fa53c109870 2024-09-19T07:23:23.103 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.179+0000 7fa53b400640 1 -- 172.21.15.159:0/1429878986 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa530032920 con 0x7fa53c109870 2024-09-19T07:23:23.103 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.179+0000 7fa53b400640 1 -- 172.21.15.159:0/1429878986 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa530030810 con 0x7fa53c109870 2024-09-19T07:23:23.103 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.179+0000 7fa543a00640 1 -- 172.21.15.159:0/1429878986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa53c109870 msgr2=0x7fa53c109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:23.103 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.179+0000 7fa543a00640 1 --2- 172.21.15.159:0/1429878986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa53c109870 0x7fa53c109c70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa530009820 tx=0x7fa53002a270 comp rx=0 tx=0).stop 2024-09-19T07:23:23.103 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.179+0000 7fa543a00640 1 -- 172.21.15.159:0/1429878986 shutdown_connections 2024-09-19T07:23:23.103 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.179+0000 7fa543a00640 1 --2- 172.21.15.159:0/1429878986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa53c109870 0x7fa53c109c70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:23.103 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.179+0000 7fa543a00640 1 -- 172.21.15.159:0/1429878986 >> 172.21.15.159:0/1429878986 conn(0x7fa53c07cf40 msgr2=0x7fa53c07d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:23.103 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.179+0000 7fa543a00640 1 -- 172.21.15.159:0/1429878986 shutdown_connections 2024-09-19T07:23:23.103 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.179+0000 7fa543a00640 1 -- 172.21.15.159:0/1429878986 wait complete. 2024-09-19T07:23:23.103 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.180+0000 7fa543a00640 1 Processor -- start 2024-09-19T07:23:23.103 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.181+0000 7fa543a00640 1 -- start start 2024-09-19T07:23:23.103 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.181+0000 7fa543a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa53c109870 0x7fa53c19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:23.103 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.181+0000 7fa543a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c10ddc0 con 0x7fa53c109870 2024-09-19T07:23:23.103 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.181+0000 7fa541000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa53c109870 0x7fa53c19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:23.104 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.181+0000 7fa541000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa53c109870 0x7fa53c19d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49372/0 (socket says 172.21.15.159:49372) 2024-09-19T07:23:23.104 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.181+0000 7fa541000640 1 -- 172.21.15.159:0/1883904554 learned_addr learned my addr 172.21.15.159:0/1883904554 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:23.104 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.182+0000 7fa541000640 1 -- 172.21.15.159:0/1883904554 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53c19dd10 con 0x7fa53c109870 2024-09-19T07:23:23.104 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.182+0000 7fa541000640 1 --2- 172.21.15.159:0/1883904554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa53c109870 0x7fa53c19d7d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa530030380 tx=0x7fa530032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:23.104 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.182+0000 7fa539600640 1 -- 172.21.15.159:0/1883904554 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa530037040 con 0x7fa53c109870 2024-09-19T07:23:23.104 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.182+0000 7fa539600640 1 -- 172.21.15.159:0/1883904554 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa530040070 con 0x7fa53c109870 2024-09-19T07:23:23.104 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.182+0000 7fa539600640 1 -- 172.21.15.159:0/1883904554 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa53003a7b0 con 0x7fa53c109870 2024-09-19T07:23:23.104 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.182+0000 7fa543a00640 1 -- 172.21.15.159:0/1883904554 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53c19dfa0 con 0x7fa53c109870 2024-09-19T07:23:23.104 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.182+0000 7fa543a00640 1 -- 172.21.15.159:0/1883904554 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa53c1a0c90 con 0x7fa53c109870 2024-09-19T07:23:23.104 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.184+0000 7fa543a00640 1 -- 172.21.15.159:0/1883904554 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa53c109cf0 con 0x7fa53c109870 2024-09-19T07:23:23.104 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.184+0000 7fa539600640 1 -- 172.21.15.159:0/1883904554 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fa53003bc00 con 0x7fa53c109870 2024-09-19T07:23:23.104 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.184+0000 7fa539600640 1 --2- 172.21.15.159:0/1883904554 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fa5240391b0 0x7fa52403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:23.104 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.184+0000 7fa539600640 1 -- 172.21.15.159:0/1883904554 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa53006c980 con 0x7fa53c109870 2024-09-19T07:23:23.105 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.185+0000 7fa53be00640 1 --2- 172.21.15.159:0/1883904554 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fa5240391b0 0x7fa52403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:23.105 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.185+0000 7fa53be00640 1 --2- 172.21.15.159:0/1883904554 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fa5240391b0 0x7fa52403b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa5280098c0 tx=0x7fa528006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:23.105 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.189+0000 7fa539600640 1 -- 172.21.15.159:0/1883904554 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa53006c2d0 con 0x7fa53c109870 2024-09-19T07:23:23.105 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.377+0000 7fa543a00640 1 -- 172.21.15.159:0/1883904554 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fa53c1a12f0 con 0x7fa53c109870 2024-09-19T07:23:23.105 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.380+0000 7fa539600640 1 -- 172.21.15.159:0/1883904554 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fa53003b7a0 con 0x7fa53c109870 2024-09-19T07:23:23.105 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.383+0000 7fa543a00640 1 -- 172.21.15.159:0/1883904554 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fa5240391b0 msgr2=0x7fa52403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:23.105 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.383+0000 7fa543a00640 1 --2- 172.21.15.159:0/1883904554 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fa5240391b0 0x7fa52403b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa5280098c0 tx=0x7fa528006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:23.105 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.383+0000 7fa543a00640 1 -- 172.21.15.159:0/1883904554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa53c109870 msgr2=0x7fa53c19d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:23.105 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.383+0000 7fa543a00640 1 --2- 172.21.15.159:0/1883904554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa53c109870 0x7fa53c19d7d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa530030380 tx=0x7fa530032670 comp rx=0 tx=0).stop 2024-09-19T07:23:23.105 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.383+0000 7fa543a00640 1 -- 172.21.15.159:0/1883904554 shutdown_connections 2024-09-19T07:23:23.105 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.383+0000 7fa543a00640 1 --2- 172.21.15.159:0/1883904554 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7fa5240391b0 0x7fa52403b670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:23.105 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.383+0000 7fa543a00640 1 --2- 172.21.15.159:0/1883904554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa53c109870 0x7fa53c19d7d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:23.106 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.383+0000 7fa543a00640 1 -- 172.21.15.159:0/1883904554 >> 172.21.15.159:0/1883904554 conn(0x7fa53c07cf40 msgr2=0x7fa53c107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:23.106 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.383+0000 7fa543a00640 1 -- 172.21.15.159:0/1883904554 shutdown_connections 2024-09-19T07:23:23.106 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:22.384+0000 7fa543a00640 1 -- 172.21.15.159:0/1883904554 wait complete. 2024-09-19T07:23:23.106 INFO:teuthology.orchestra.run.smithi159.stdout:Enabling the dashboard module... 2024-09-19T07:23:23.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:23 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' 2024-09-19T07:23:23.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:23 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1883904554' entity='client.admin' 2024-09-19T07:23:24.497 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:24 smithi159 ceph-mon[30472]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:23:24.497 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:24 smithi159 ceph-mon[30472]: from='mgr.14116 172.21.15.159:0/1311032044' entity='mgr.x' 2024-09-19T07:23:24.497 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:24 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1614948011' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T07:23:25.137 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:24 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: ignoring --setuser ceph since I am not root 2024-09-19T07:23:25.137 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:24 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: ignoring --setgroup ceph since I am not root 2024-09-19T07:23:25.137 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:24 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:24.950+0000 7f08e97ef040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T07:23:25.137 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:25 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:25.020+0000 7f08e97ef040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T07:23:25.460 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:25 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:25.205+0000 7f08e97ef040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T07:23:25.460 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:25 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:25.289+0000 7f08e97ef040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T07:23:25.460 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:25 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:25.459+0000 7f08e97ef040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:23:25.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.660+0000 7f7e59a00640 1 Processor -- start 2024-09-19T07:23:25.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.660+0000 7f7e59a00640 1 -- start start 2024-09-19T07:23:25.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.660+0000 7f7e59a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e54108150 0x7f7e5410a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:25.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.660+0000 7f7e59a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5407bdf0 con 0x7f7e54108150 2024-09-19T07:23:25.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.661+0000 7f7e52a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e54108150 0x7f7e5410a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:25.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.661+0000 7f7e52a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e54108150 0x7f7e5410a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49382/0 (socket says 172.21.15.159:49382) 2024-09-19T07:23:25.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.661+0000 7f7e52a00640 1 -- 172.21.15.159:0/400911539 learned_addr learned my addr 172.21.15.159:0/400911539 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:25.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.661+0000 7f7e52a00640 1 -- 172.21.15.159:0/400911539 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5410aa80 con 0x7f7e54108150 2024-09-19T07:23:25.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.662+0000 7f7e52a00640 1 --2- 172.21.15.159:0/400911539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e54108150 0x7f7e5410a540 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009820 tx=0x7f7e4402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3edcf0f33b58ebc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:25.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.663+0000 7f7e51600640 1 -- 172.21.15.159:0/400911539 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e44037070 con 0x7f7e54108150 2024-09-19T07:23:25.619 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.663+0000 7f7e51600640 1 -- 172.21.15.159:0/400911539 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e44032920 con 0x7f7e54108150 2024-09-19T07:23:25.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.663+0000 7f7e51600640 1 -- 172.21.15.159:0/400911539 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e44030810 con 0x7f7e54108150 2024-09-19T07:23:25.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.664+0000 7f7e59a00640 1 -- 172.21.15.159:0/400911539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e54108150 msgr2=0x7f7e5410a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:25.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.664+0000 7f7e59a00640 1 --2- 172.21.15.159:0/400911539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e54108150 0x7f7e5410a540 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009820 tx=0x7f7e4402a270 comp rx=0 tx=0).stop 2024-09-19T07:23:25.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.664+0000 7f7e59a00640 1 -- 172.21.15.159:0/400911539 shutdown_connections 2024-09-19T07:23:25.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.664+0000 7f7e59a00640 1 --2- 172.21.15.159:0/400911539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e54108150 0x7f7e5410a540 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:25.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.664+0000 7f7e59a00640 1 -- 172.21.15.159:0/400911539 >> 172.21.15.159:0/400911539 conn(0x7f7e54101db0 msgr2=0x7f7e541041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:25.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.664+0000 7f7e59a00640 1 -- 172.21.15.159:0/400911539 shutdown_connections 2024-09-19T07:23:25.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.664+0000 7f7e59a00640 1 -- 172.21.15.159:0/400911539 wait complete. 2024-09-19T07:23:25.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.665+0000 7f7e59a00640 1 Processor -- start 2024-09-19T07:23:25.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.665+0000 7f7e59a00640 1 -- start start 2024-09-19T07:23:25.620 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.665+0000 7f7e59a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e54108150 0x7f7e5419d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:25.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.665+0000 7f7e59a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5410baa0 con 0x7f7e54108150 2024-09-19T07:23:25.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.666+0000 7f7e52a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e54108150 0x7f7e5419d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:25.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.666+0000 7f7e52a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e54108150 0x7f7e5419d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49390/0 (socket says 172.21.15.159:49390) 2024-09-19T07:23:25.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.666+0000 7f7e52a00640 1 -- 172.21.15.159:0/1614948011 learned_addr learned my addr 172.21.15.159:0/1614948011 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:25.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.666+0000 7f7e52a00640 1 -- 172.21.15.159:0/1614948011 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5419dd70 con 0x7f7e54108150 2024-09-19T07:23:25.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.666+0000 7f7e52a00640 1 --2- 172.21.15.159:0/1614948011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e54108150 0x7f7e5419d830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7e440097f0 tx=0x7f7e440327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:25.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.666+0000 7f7e4b400640 1 -- 172.21.15.159:0/1614948011 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e44037040 con 0x7f7e54108150 2024-09-19T07:23:25.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.667+0000 7f7e4b400640 1 -- 172.21.15.159:0/1614948011 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e44030cc0 con 0x7f7e54108150 2024-09-19T07:23:25.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.667+0000 7f7e59a00640 1 -- 172.21.15.159:0/1614948011 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5419e000 con 0x7f7e54108150 2024-09-19T07:23:25.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.667+0000 7f7e4b400640 1 -- 172.21.15.159:0/1614948011 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e4403a6b0 con 0x7f7e54108150 2024-09-19T07:23:25.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.667+0000 7f7e59a00640 1 -- 172.21.15.159:0/1614948011 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e541a0cf0 con 0x7f7e54108150 2024-09-19T07:23:25.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.668+0000 7f7e4b400640 1 -- 172.21.15.159:0/1614948011 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f7e4403a8a0 con 0x7f7e54108150 2024-09-19T07:23:25.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.668+0000 7f7e4b400640 1 --2- 172.21.15.159:0/1614948011 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f7e380391b0 0x7f7e3803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:25.621 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.668+0000 7f7e4b400640 1 -- 172.21.15.159:0/1614948011 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7e4406c9e0 con 0x7f7e54108150 2024-09-19T07:23:25.622 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.668+0000 7f7e59a00640 1 -- 172.21.15.159:0/1614948011 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e541085b0 con 0x7f7e54108150 2024-09-19T07:23:25.622 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.668+0000 7f7e52000640 1 --2- 172.21.15.159:0/1614948011 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f7e380391b0 0x7f7e3803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:25.622 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.669+0000 7f7e52000640 1 --2- 172.21.15.159:0/1614948011 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f7e380391b0 0x7f7e3803b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7e400098c0 tx=0x7f7e40006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:25.622 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.674+0000 7f7e4b400640 1 -- 172.21.15.159:0/1614948011 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7e4406c330 con 0x7f7e54108150 2024-09-19T07:23:25.622 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:23.922+0000 7f7e59a00640 1 -- 172.21.15.159:0/1614948011 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f7e541a11b0 con 0x7f7e54108150 2024-09-19T07:23:25.622 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:24.706+0000 7f7e4b400640 1 -- 172.21.15.159:0/1614948011 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f7e44044920 con 0x7f7e54108150 2024-09-19T07:23:25.622 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:24.708+0000 7f7e4b400640 1 -- 172.21.15.159:0/1614948011 <== mon.0 v2:172.21.15.159:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f7e44044400 con 0x7f7e54108150 2024-09-19T07:23:25.622 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:24.710+0000 7f7e59a00640 1 -- 172.21.15.159:0/1614948011 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f7e380391b0 msgr2=0x7f7e3803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:25.622 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:24.710+0000 7f7e59a00640 1 --2- 172.21.15.159:0/1614948011 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f7e380391b0 0x7f7e3803b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7e400098c0 tx=0x7f7e40006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:25.622 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:24.710+0000 7f7e59a00640 1 -- 172.21.15.159:0/1614948011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e54108150 msgr2=0x7f7e5419d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:25.622 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:24.710+0000 7f7e59a00640 1 --2- 172.21.15.159:0/1614948011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e54108150 0x7f7e5419d830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7e440097f0 tx=0x7f7e440327b0 comp rx=0 tx=0).stop 2024-09-19T07:23:25.622 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:24.710+0000 7f7e59a00640 1 -- 172.21.15.159:0/1614948011 shutdown_connections 2024-09-19T07:23:25.622 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:24.710+0000 7f7e59a00640 1 --2- 172.21.15.159:0/1614948011 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f7e380391b0 0x7f7e3803b670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:25.623 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:24.710+0000 7f7e59a00640 1 --2- 172.21.15.159:0/1614948011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e54108150 0x7f7e5419d830 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:25.623 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:24.711+0000 7f7e59a00640 1 -- 172.21.15.159:0/1614948011 >> 172.21.15.159:0/1614948011 conn(0x7f7e54101db0 msgr2=0x7f7e54102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:25.623 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:24.711+0000 7f7e59a00640 1 -- 172.21.15.159:0/1614948011 shutdown_connections 2024-09-19T07:23:25.623 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:24.711+0000 7f7e59a00640 1 -- 172.21.15.159:0/1614948011 wait complete. 2024-09-19T07:23:25.755 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:25 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1614948011' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T07:23:25.755 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:25 smithi159 ceph-mon[30472]: mgrmap e9: x(active, since 21s) 2024-09-19T07:23:25.755 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:25 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:25.629+0000 7f08e97ef040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T07:23:26.278 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:26 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:26.013+0000 7f08e97ef040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T07:23:26.278 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:26 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:26.086+0000 7f08e97ef040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T07:23:26.548 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:26 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:26.302+0000 7f08e97ef040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T07:23:26.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:26 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3793189981' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T07:23:27.337 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout { 2024-09-19T07:23:27.337 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T07:23:27.337 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T07:23:27.338 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T07:23:27.338 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T07:23:27.338 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout } 2024-09-19T07:23:27.338 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.301+0000 7f8eba800640 1 Processor -- start 2024-09-19T07:23:27.338 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.302+0000 7f8eba800640 1 -- start start 2024-09-19T07:23:27.338 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.302+0000 7f8eba800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8eb407de60 0x7f8eb407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:27.338 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.302+0000 7f8eba800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb407c800 con 0x7f8eb407de60 2024-09-19T07:23:27.338 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.302+0000 7f8eb9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8eb407de60 0x7f8eb407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:27.338 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.302+0000 7f8eb9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8eb407de60 0x7f8eb407c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49416/0 (socket says 172.21.15.159:49416) 2024-09-19T07:23:27.338 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.302+0000 7f8eb9400640 1 -- 172.21.15.159:0/2723971343 learned_addr learned my addr 172.21.15.159:0/2723971343 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:27.338 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.303+0000 7f8eb9400640 1 -- 172.21.15.159:0/2723971343 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb407c980 con 0x7f8eb407de60 2024-09-19T07:23:27.338 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.304+0000 7f8eb9400640 1 --2- 172.21.15.159:0/2723971343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8eb407de60 0x7f8eb407c2c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009820 tx=0x7f8ea802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3122ec3b132e798 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:27.339 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.304+0000 7f8eb3e00640 1 -- 172.21.15.159:0/2723971343 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ea8037070 con 0x7f8eb407de60 2024-09-19T07:23:27.339 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.304+0000 7f8eb3e00640 1 -- 172.21.15.159:0/2723971343 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ea8032920 con 0x7f8eb407de60 2024-09-19T07:23:27.339 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.304+0000 7f8eb3e00640 1 -- 172.21.15.159:0/2723971343 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ea8030810 con 0x7f8eb407de60 2024-09-19T07:23:27.339 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.305+0000 7f8eba800640 1 -- 172.21.15.159:0/2723971343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8eb407de60 msgr2=0x7f8eb407c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:27.339 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.305+0000 7f8eba800640 1 --2- 172.21.15.159:0/2723971343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8eb407de60 0x7f8eb407c2c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009820 tx=0x7f8ea802a270 comp rx=0 tx=0).stop 2024-09-19T07:23:27.339 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.305+0000 7f8eba800640 1 -- 172.21.15.159:0/2723971343 shutdown_connections 2024-09-19T07:23:27.339 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.305+0000 7f8eba800640 1 --2- 172.21.15.159:0/2723971343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8eb407de60 0x7f8eb407c2c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:27.339 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.305+0000 7f8eba800640 1 -- 172.21.15.159:0/2723971343 >> 172.21.15.159:0/2723971343 conn(0x7f8eb4102c10 msgr2=0x7f8eb4105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:27.339 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.305+0000 7f8eba800640 1 -- 172.21.15.159:0/2723971343 shutdown_connections 2024-09-19T07:23:27.339 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.305+0000 7f8eba800640 1 -- 172.21.15.159:0/2723971343 wait complete. 2024-09-19T07:23:27.340 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.306+0000 7f8eba800640 1 Processor -- start 2024-09-19T07:23:27.340 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.306+0000 7f8eba800640 1 -- start start 2024-09-19T07:23:27.340 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.307+0000 7f8eba800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8eb407de60 0x7f8eb419d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:27.340 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.307+0000 7f8eba800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb410da90 con 0x7f8eb407de60 2024-09-19T07:23:27.340 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.307+0000 7f8eb9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8eb407de60 0x7f8eb419d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:27.340 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.307+0000 7f8eb9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8eb407de60 0x7f8eb419d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49420/0 (socket says 172.21.15.159:49420) 2024-09-19T07:23:27.340 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.307+0000 7f8eb9400640 1 -- 172.21.15.159:0/3793189981 learned_addr learned my addr 172.21.15.159:0/3793189981 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:27.340 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.307+0000 7f8eb9400640 1 -- 172.21.15.159:0/3793189981 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb419dc00 con 0x7f8eb407de60 2024-09-19T07:23:27.340 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.308+0000 7f8eb9400640 1 --2- 172.21.15.159:0/3793189981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8eb407de60 0x7f8eb419d6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8030420 tx=0x7f8ea8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:27.341 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.308+0000 7f8eb2000640 1 -- 172.21.15.159:0/3793189981 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ea8037040 con 0x7f8eb407de60 2024-09-19T07:23:27.341 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.308+0000 7f8eb2000640 1 -- 172.21.15.159:0/3793189981 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ea8040070 con 0x7f8eb407de60 2024-09-19T07:23:27.341 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.308+0000 7f8eb2000640 1 -- 172.21.15.159:0/3793189981 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ea803a6e0 con 0x7f8eb407de60 2024-09-19T07:23:27.341 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.308+0000 7f8eba800640 1 -- 172.21.15.159:0/3793189981 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb419de90 con 0x7f8eb407de60 2024-09-19T07:23:27.341 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.308+0000 7f8eba800640 1 -- 172.21.15.159:0/3793189981 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb41a0b80 con 0x7f8eb407de60 2024-09-19T07:23:27.341 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.309+0000 7f8eb2000640 1 -- 172.21.15.159:0/3793189981 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f8ea803a990 con 0x7f8eb407de60 2024-09-19T07:23:27.341 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.309+0000 7f8ea7400640 1 -- 172.21.15.159:0/3793189981 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb407c340 con 0x7f8eb407de60 2024-09-19T07:23:27.341 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.309+0000 7f8eb2000640 1 --2- 172.21.15.159:0/3793189981 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f8e9c039260 0x7f8e9c03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:27.341 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.310+0000 7f8eb2000640 1 -- 172.21.15.159:0/3793189981 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8ea806cf10 con 0x7f8eb407de60 2024-09-19T07:23:27.342 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.310+0000 7f8eb8a00640 1 -- 172.21.15.159:0/3793189981 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f8e9c039260 msgr2=0x7f8e9c03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.159:6800/2966947684 2024-09-19T07:23:27.342 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.310+0000 7f8eb8a00640 1 --2- 172.21.15.159:0/3793189981 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f8e9c039260 0x7f8e9c03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:23:27.342 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.315+0000 7f8eb2000640 1 -- 172.21.15.159:0/3793189981 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8ea80328c0 con 0x7f8eb407de60 2024-09-19T07:23:27.342 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.511+0000 7f8eb8a00640 1 -- 172.21.15.159:0/3793189981 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f8e9c039260 msgr2=0x7f8e9c03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.159:6800/2966947684 2024-09-19T07:23:27.342 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.511+0000 7f8eb8a00640 1 --2- 172.21.15.159:0/3793189981 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f8e9c039260 0x7f8e9c03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T07:23:27.342 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.546+0000 7f8ea7400640 1 -- 172.21.15.159:0/3793189981 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f8eb4062630 con 0x7f8eb407de60 2024-09-19T07:23:27.342 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.547+0000 7f8eb2000640 1 -- 172.21.15.159:0/3793189981 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f8ea806f020 con 0x7f8eb407de60 2024-09-19T07:23:27.342 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.549+0000 7f8ea7400640 1 -- 172.21.15.159:0/3793189981 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f8e9c039260 msgr2=0x7f8e9c03b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:23:27.342 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.549+0000 7f8ea7400640 1 --2- 172.21.15.159:0/3793189981 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f8e9c039260 0x7f8e9c03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:27.342 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.549+0000 7f8ea7400640 1 -- 172.21.15.159:0/3793189981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8eb407de60 msgr2=0x7f8eb419d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:27.342 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.549+0000 7f8ea7400640 1 --2- 172.21.15.159:0/3793189981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8eb407de60 0x7f8eb419d6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8030420 tx=0x7f8ea8032670 comp rx=0 tx=0).stop 2024-09-19T07:23:27.342 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.550+0000 7f8ea7400640 1 -- 172.21.15.159:0/3793189981 shutdown_connections 2024-09-19T07:23:27.343 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.550+0000 7f8ea7400640 1 --2- 172.21.15.159:0/3793189981 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f8e9c039260 0x7f8e9c03b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:27.343 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.550+0000 7f8ea7400640 1 --2- 172.21.15.159:0/3793189981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8eb407de60 0x7f8eb419d6c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:27.343 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.550+0000 7f8ea7400640 1 -- 172.21.15.159:0/3793189981 >> 172.21.15.159:0/3793189981 conn(0x7f8eb4102c10 msgr2=0x7f8eb41037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:27.343 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.550+0000 7f8ea7400640 1 -- 172.21.15.159:0/3793189981 shutdown_connections 2024-09-19T07:23:27.343 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:26.550+0000 7f8ea7400640 1 -- 172.21.15.159:0/3793189981 wait complete. 2024-09-19T07:23:27.343 INFO:teuthology.orchestra.run.smithi159.stdout:Waiting for the mgr to restart... 2024-09-19T07:23:27.343 INFO:teuthology.orchestra.run.smithi159.stdout:Waiting for mgr epoch 9... 2024-09-19T07:23:27.636 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:27 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:27.436+0000 7f08e97ef040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T07:23:27.636 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:27 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:27.516+0000 7f08e97ef040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T07:23:28.735 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:28 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:28.438+0000 7f08e97ef040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T07:23:28.735 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:28 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:28.573+0000 7f08e97ef040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T07:23:28.735 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:28 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:28.733+0000 7f08e97ef040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:23:29.057 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:28 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:28.807+0000 7f08e97ef040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T07:23:29.057 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:28 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:28.976+0000 7f08e97ef040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T07:23:29.057 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:29 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:29.056+0000 7f08e97ef040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T07:23:29.349 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:29 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:29.126+0000 7f08e97ef040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T07:23:29.349 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:29 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:29.303+0000 7f08e97ef040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T07:23:29.621 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:29 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:29.380+0000 7f08e97ef040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T07:23:29.622 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:29 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:29.457+0000 7f08e97ef040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T07:23:29.622 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:29 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:29.551+0000 7f08e97ef040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T07:23:29.622 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:29 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:29.620+0000 7f08e97ef040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T07:23:29.886 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:29 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:29.691+0000 7f08e97ef040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T07:23:30.887 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:30 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:30.594+0000 7f08e97ef040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T07:23:30.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:30 smithi159 ceph-mon[30472]: Active manager daemon x restarted 2024-09-19T07:23:30.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:30 smithi159 ceph-mon[30472]: Activating manager daemon x 2024-09-19T07:23:30.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:30 smithi159 ceph-mon[30472]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T07:23:30.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:30 smithi159 ceph-mon[30472]: mgrmap e10: x(active, starting, since 0.00398825s) 2024-09-19T07:23:30.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:23:30.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T07:23:30.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T07:23:30.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T07:23:30.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T07:23:30.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:30 smithi159 ceph-mon[30472]: Manager daemon x is now available 2024-09-19T07:23:30.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:23:31.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T07:23:31.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T07:23:31.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:31 smithi159 ceph-mon[30472]: mgrmap e11: x(active, since 1.00707s) 2024-09-19T07:23:32.387 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:32.022+0000 7f0846000640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:07:23:32] ENGINE Error in HTTPServer.serve 2024-09-19T07:23:32.387 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: Traceback (most recent call last): 2024-09-19T07:23:32.387 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T07:23:32.387 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: self._connections.run(self.expiration_interval) 2024-09-19T07:23:32.387 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T07:23:32.388 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: self._run(expiration_interval) 2024-09-19T07:23:32.388 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T07:23:32.388 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T07:23:32.388 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T07:23:32.388 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T07:23:32.388 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T07:23:32.388 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: s = self.context.wrap_socket( 2024-09-19T07:23:32.388 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T07:23:32.388 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: return self.sslsocket_class._create( 2024-09-19T07:23:32.388 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T07:23:32.388 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: self.do_handshake() 2024-09-19T07:23:32.388 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T07:23:32.389 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: self._sslobj.do_handshake() 2024-09-19T07:23:32.389 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T07:23:32.389 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:23:32.887 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout { 2024-09-19T07:23:32.887 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T07:23:32.887 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T07:23:32.887 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout } 2024-09-19T07:23:32.888 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.936+0000 7f6035000640 1 Processor -- start 2024-09-19T07:23:32.888 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.937+0000 7f6035000640 1 -- start start 2024-09-19T07:23:32.888 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.937+0000 7f6035000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f603007de80 0x7f603007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:32.888 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.937+0000 7f6035000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f603007c8b0 con 0x7f603007de80 2024-09-19T07:23:32.888 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.937+0000 7f602f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f603007de80 0x7f603007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:32.888 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.937+0000 7f602f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f603007de80 0x7f603007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49434/0 (socket says 172.21.15.159:49434) 2024-09-19T07:23:32.888 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.937+0000 7f602f400640 1 -- 172.21.15.159:0/2486179149 learned_addr learned my addr 172.21.15.159:0/2486179149 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:32.888 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.938+0000 7f602f400640 1 -- 172.21.15.159:0/2486179149 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603007ca30 con 0x7f603007de80 2024-09-19T07:23:32.888 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.939+0000 7f602f400640 1 --2- 172.21.15.159:0/2486179149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f603007de80 0x7f603007c2e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6020009820 tx=0x7f602002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae8fc25606166c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:32.888 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.939+0000 7f602e000640 1 -- 172.21.15.159:0/2486179149 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6020037070 con 0x7f603007de80 2024-09-19T07:23:32.888 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.939+0000 7f602e000640 1 -- 172.21.15.159:0/2486179149 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6020032920 con 0x7f603007de80 2024-09-19T07:23:32.888 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.939+0000 7f602e000640 1 -- 172.21.15.159:0/2486179149 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6020030810 con 0x7f603007de80 2024-09-19T07:23:32.888 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.940+0000 7f6035000640 1 -- 172.21.15.159:0/2486179149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f603007de80 msgr2=0x7f603007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:32.889 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.940+0000 7f6035000640 1 --2- 172.21.15.159:0/2486179149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f603007de80 0x7f603007c2e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6020009820 tx=0x7f602002a270 comp rx=0 tx=0).stop 2024-09-19T07:23:32.889 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.940+0000 7f6035000640 1 -- 172.21.15.159:0/2486179149 shutdown_connections 2024-09-19T07:23:32.889 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.940+0000 7f6035000640 1 --2- 172.21.15.159:0/2486179149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f603007de80 0x7f603007c2e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:32.889 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.940+0000 7f6035000640 1 -- 172.21.15.159:0/2486179149 >> 172.21.15.159:0/2486179149 conn(0x7f6030102c10 msgr2=0x7f6030105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:32.889 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.940+0000 7f6035000640 1 -- 172.21.15.159:0/2486179149 shutdown_connections 2024-09-19T07:23:32.889 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.940+0000 7f6035000640 1 -- 172.21.15.159:0/2486179149 wait complete. 2024-09-19T07:23:32.889 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.941+0000 7f6035000640 1 Processor -- start 2024-09-19T07:23:32.889 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.941+0000 7f6035000640 1 -- start start 2024-09-19T07:23:32.889 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.941+0000 7f6035000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f603007de80 0x7f603019d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:32.889 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.941+0000 7f6035000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f603010dad0 con 0x7f603007de80 2024-09-19T07:23:32.889 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.942+0000 7f602f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f603007de80 0x7f603019d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:32.889 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.942+0000 7f602f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f603007de80 0x7f603019d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49446/0 (socket says 172.21.15.159:49446) 2024-09-19T07:23:32.890 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.942+0000 7f602f400640 1 -- 172.21.15.159:0/1391360143 learned_addr learned my addr 172.21.15.159:0/1391360143 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:32.890 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.942+0000 7f602f400640 1 -- 172.21.15.159:0/1391360143 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603019dc20 con 0x7f603007de80 2024-09-19T07:23:32.890 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.942+0000 7f602f400640 1 --2- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f603007de80 0x7f603019d6e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f60200097f0 tx=0x7f60200327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:32.890 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.943+0000 7f6027e00640 1 -- 172.21.15.159:0/1391360143 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6020037040 con 0x7f603007de80 2024-09-19T07:23:32.890 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.943+0000 7f6027e00640 1 -- 172.21.15.159:0/1391360143 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6020030c40 con 0x7f603007de80 2024-09-19T07:23:32.890 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.943+0000 7f6035000640 1 -- 172.21.15.159:0/1391360143 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603019deb0 con 0x7f603007de80 2024-09-19T07:23:32.890 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.943+0000 7f6027e00640 1 -- 172.21.15.159:0/1391360143 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f602003a5d0 con 0x7f603007de80 2024-09-19T07:23:32.890 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.943+0000 7f6035000640 1 -- 172.21.15.159:0/1391360143 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60301a0ba0 con 0x7f603007de80 2024-09-19T07:23:32.890 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.944+0000 7f6027e00640 1 -- 172.21.15.159:0/1391360143 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f602003b3f0 con 0x7f603007de80 2024-09-19T07:23:32.890 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.945+0000 7f6027e00640 1 --2- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f6014039260 0x7f601403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:32.890 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.945+0000 7f6027e00640 1 -- 172.21.15.159:0/1391360143 --> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f601403be30 con 0x7f6014039260 2024-09-19T07:23:32.890 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.945+0000 7f6027e00640 1 -- 172.21.15.159:0/1391360143 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f602006c7e0 con 0x7f603007de80 2024-09-19T07:23:32.891 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.945+0000 7f602ea00640 1 -- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f6014039260 msgr2=0x7f601403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.159:6800/2966947684 2024-09-19T07:23:32.891 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:27.945+0000 7f602ea00640 1 --2- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f6014039260 0x7f601403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:23:32.891 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:28.146+0000 7f602ea00640 1 -- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f6014039260 msgr2=0x7f601403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.159:6800/2966947684 2024-09-19T07:23:32.891 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:28.146+0000 7f602ea00640 1 --2- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f6014039260 0x7f601403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T07:23:32.891 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:28.547+0000 7f602ea00640 1 -- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f6014039260 msgr2=0x7f601403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.159:6800/2966947684 2024-09-19T07:23:32.891 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:28.547+0000 7f602ea00640 1 --2- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f6014039260 0x7f601403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T07:23:32.891 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:29.348+0000 7f602ea00640 1 -- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f6014039260 msgr2=0x7f601403b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.159:6800/2966947684 2024-09-19T07:23:32.891 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:29.348+0000 7f602ea00640 1 --2- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f6014039260 0x7f601403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T07:23:32.891 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:30.598+0000 7f6027e00640 1 -- 172.21.15.159:0/1391360143 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f602006b210 con 0x7f603007de80 2024-09-19T07:23:32.891 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:30.598+0000 7f6027e00640 1 -- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f6014039260 msgr2=0x7f601403b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:23:32.891 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:30.598+0000 7f6027e00640 1 --2- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f6014039260 0x7f601403b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:32.891 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.600+0000 7f6027e00640 1 -- 172.21.15.159:0/1391360143 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f6020036170 con 0x7f603007de80 2024-09-19T07:23:32.891 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.600+0000 7f6027e00640 1 --2- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f601403cd20 0x7f601403f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:32.892 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.600+0000 7f6027e00640 1 -- 172.21.15.159:0/1391360143 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6020041e70 con 0x7f601403cd20 2024-09-19T07:23:32.892 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.601+0000 7f602ea00640 1 --2- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f601403cd20 0x7f601403f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:32.892 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.602+0000 7f602ea00640 1 --2- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f601403cd20 0x7f601403f110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f601c003980 tx=0x7f601c0092b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:32.892 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.603+0000 7f6027e00640 1 -- 172.21.15.159:0/1391360143 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f6020041e70 con 0x7f601403cd20 2024-09-19T07:23:32.892 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.610+0000 7f6035000640 1 -- 172.21.15.159:0/1391360143 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f603007c360 con 0x7f601403cd20 2024-09-19T07:23:32.892 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.611+0000 7f6027e00640 1 -- 172.21.15.159:0/1391360143 <== mgr.14148 v2:172.21.15.159:6800/4243905339 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f603007c360 con 0x7f601403cd20 2024-09-19T07:23:32.892 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.611+0000 7f6035000640 1 -- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f601403cd20 msgr2=0x7f601403f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:32.892 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.611+0000 7f6035000640 1 --2- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f601403cd20 0x7f601403f110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f601c003980 tx=0x7f601c0092b0 comp rx=0 tx=0).stop 2024-09-19T07:23:32.892 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.611+0000 7f6035000640 1 -- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f603007de80 msgr2=0x7f603019d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:32.892 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.611+0000 7f6035000640 1 --2- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f603007de80 0x7f603019d6e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f60200097f0 tx=0x7f60200327b0 comp rx=0 tx=0).stop 2024-09-19T07:23:32.892 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.611+0000 7f6035000640 1 -- 172.21.15.159:0/1391360143 shutdown_connections 2024-09-19T07:23:32.892 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.611+0000 7f6035000640 1 --2- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f601403cd20 0x7f601403f110 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:32.892 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.611+0000 7f6035000640 1 --2- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:6800/2966947684,v1:172.21.15.159:6801/2966947684] conn(0x7f6014039260 0x7f601403b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:32.893 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.611+0000 7f6035000640 1 --2- 172.21.15.159:0/1391360143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f603007de80 0x7f603019d6e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:32.893 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.611+0000 7f6035000640 1 -- 172.21.15.159:0/1391360143 >> 172.21.15.159:0/1391360143 conn(0x7f6030102c10 msgr2=0x7f6030103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:32.893 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.611+0000 7f6035000640 1 -- 172.21.15.159:0/1391360143 shutdown_connections 2024-09-19T07:23:32.893 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:31.612+0000 7f6035000640 1 -- 172.21.15.159:0/1391360143 wait complete. 2024-09-19T07:23:32.893 INFO:teuthology.orchestra.run.smithi159.stdout:mgr epoch 9 is available 2024-09-19T07:23:32.893 INFO:teuthology.orchestra.run.smithi159.stdout:Generating a dashboard self-signed certificate... 2024-09-19T07:23:33.024 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-mon[30472]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T07:23:33.024 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:32 smithi159 ceph-mon[30472]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T07:23:33.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:33 smithi159 ceph-mon[30472]: [19/Sep/2024:07:23:31] ENGINE Bus STARTING 2024-09-19T07:23:33.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:33 smithi159 ceph-mon[30472]: [19/Sep/2024:07:23:32] ENGINE Serving on https://172.21.15.159:7150 2024-09-19T07:23:33.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:33 smithi159 ceph-mon[30472]: [19/Sep/2024:07:23:32] ENGINE Error in HTTPServer.serve 2024-09-19T07:23:33.887 INFO:journalctl@ceph.mon.a.smithi159.stdout: Traceback (most recent call last): 2024-09-19T07:23:33.887 INFO:journalctl@ceph.mon.a.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T07:23:33.887 INFO:journalctl@ceph.mon.a.smithi159.stdout: self._connections.run(self.expiration_interval) 2024-09-19T07:23:33.887 INFO:journalctl@ceph.mon.a.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T07:23:33.888 INFO:journalctl@ceph.mon.a.smithi159.stdout: self._run(expiration_interval) 2024-09-19T07:23:33.888 INFO:journalctl@ceph.mon.a.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T07:23:33.888 INFO:journalctl@ceph.mon.a.smithi159.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T07:23:33.888 INFO:journalctl@ceph.mon.a.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T07:23:33.888 INFO:journalctl@ceph.mon.a.smithi159.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T07:23:33.888 INFO:journalctl@ceph.mon.a.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T07:23:33.888 INFO:journalctl@ceph.mon.a.smithi159.stdout: s = self.context.wrap_socket( 2024-09-19T07:23:33.888 INFO:journalctl@ceph.mon.a.smithi159.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T07:23:33.888 INFO:journalctl@ceph.mon.a.smithi159.stdout: return self.sslsocket_class._create( 2024-09-19T07:23:33.888 INFO:journalctl@ceph.mon.a.smithi159.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T07:23:33.889 INFO:journalctl@ceph.mon.a.smithi159.stdout: self.do_handshake() 2024-09-19T07:23:33.889 INFO:journalctl@ceph.mon.a.smithi159.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T07:23:33.889 INFO:journalctl@ceph.mon.a.smithi159.stdout: self._sslobj.do_handshake() 2024-09-19T07:23:33.889 INFO:journalctl@ceph.mon.a.smithi159.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T07:23:33.889 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:33 smithi159 ceph-mon[30472]: [19/Sep/2024:07:23:32] ENGINE Serving on http://172.21.15.159:8765 2024-09-19T07:23:33.889 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:33 smithi159 ceph-mon[30472]: [19/Sep/2024:07:23:32] ENGINE Bus STARTED 2024-09-19T07:23:33.889 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:33 smithi159 ceph-mon[30472]: mgrmap e12: x(active, since 2s) 2024-09-19T07:23:33.889 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:34.375 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T07:23:34.375 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.421+0000 7f1d02200640 1 Processor -- start 2024-09-19T07:23:34.375 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.423+0000 7f1d02200640 1 -- start start 2024-09-19T07:23:34.375 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.423+0000 7f1d02200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cfc107440 0x7f1cfc107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:34.376 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.423+0000 7f1d02200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc107e10 con 0x7f1cfc107440 2024-09-19T07:23:34.376 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.423+0000 7f1d00e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cfc107440 0x7f1cfc107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:34.376 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.423+0000 7f1d00e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cfc107440 0x7f1cfc107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50306/0 (socket says 172.21.15.159:50306) 2024-09-19T07:23:34.376 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.423+0000 7f1d00e00640 1 -- 172.21.15.159:0/583177817 learned_addr learned my addr 172.21.15.159:0/583177817 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:34.376 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.424+0000 7f1d00e00640 1 -- 172.21.15.159:0/583177817 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cfc108640 con 0x7f1cfc107440 2024-09-19T07:23:34.376 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.425+0000 7f1d00e00640 1 --2- 172.21.15.159:0/583177817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cfc107440 0x7f1cfc107840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009820 tx=0x7f1cf002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51774bc418428cd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:34.376 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.425+0000 7f1cfb400640 1 -- 172.21.15.159:0/583177817 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cf0037070 con 0x7f1cfc107440 2024-09-19T07:23:34.376 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.425+0000 7f1cfb400640 1 -- 172.21.15.159:0/583177817 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf0032920 con 0x7f1cfc107440 2024-09-19T07:23:34.376 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.425+0000 7f1cfb400640 1 -- 172.21.15.159:0/583177817 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cf0030810 con 0x7f1cfc107440 2024-09-19T07:23:34.376 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.426+0000 7f1d02200640 1 -- 172.21.15.159:0/583177817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cfc107440 msgr2=0x7f1cfc107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:34.376 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.426+0000 7f1d02200640 1 --2- 172.21.15.159:0/583177817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cfc107440 0x7f1cfc107840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009820 tx=0x7f1cf002a270 comp rx=0 tx=0).stop 2024-09-19T07:23:34.376 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.426+0000 7f1d02200640 1 -- 172.21.15.159:0/583177817 shutdown_connections 2024-09-19T07:23:34.376 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.426+0000 7f1d02200640 1 --2- 172.21.15.159:0/583177817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cfc107440 0x7f1cfc107840 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:34.377 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.426+0000 7f1d02200640 1 -- 172.21.15.159:0/583177817 >> 172.21.15.159:0/583177817 conn(0x7f1cfc102c10 msgr2=0x7f1cfc105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:34.377 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.426+0000 7f1d02200640 1 -- 172.21.15.159:0/583177817 shutdown_connections 2024-09-19T07:23:34.377 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.426+0000 7f1d02200640 1 -- 172.21.15.159:0/583177817 wait complete. 2024-09-19T07:23:34.377 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.427+0000 7f1d02200640 1 Processor -- start 2024-09-19T07:23:34.377 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.427+0000 7f1d02200640 1 -- start start 2024-09-19T07:23:34.377 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.428+0000 7f1d02200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cfc107440 0x7f1cfc07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:34.377 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.428+0000 7f1d02200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc10b9f0 con 0x7f1cfc107440 2024-09-19T07:23:34.377 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.428+0000 7f1d00e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cfc107440 0x7f1cfc07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:34.377 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.428+0000 7f1d00e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cfc107440 0x7f1cfc07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50314/0 (socket says 172.21.15.159:50314) 2024-09-19T07:23:34.377 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.428+0000 7f1d00e00640 1 -- 172.21.15.159:0/1097006010 learned_addr learned my addr 172.21.15.159:0/1097006010 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:34.377 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.428+0000 7f1d00e00640 1 -- 172.21.15.159:0/1097006010 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cfc07e0b0 con 0x7f1cfc107440 2024-09-19T07:23:34.377 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.428+0000 7f1d00e00640 1 --2- 172.21.15.159:0/1097006010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cfc107440 0x7f1cfc07db70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00097f0 tx=0x7f1cf00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:34.377 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.429+0000 7f1cf9600640 1 -- 172.21.15.159:0/1097006010 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cf0037040 con 0x7f1cfc107440 2024-09-19T07:23:34.378 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.429+0000 7f1cf9600640 1 -- 172.21.15.159:0/1097006010 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf0030c40 con 0x7f1cfc107440 2024-09-19T07:23:34.378 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.429+0000 7f1cf9600640 1 -- 172.21.15.159:0/1097006010 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cf003a5b0 con 0x7f1cfc107440 2024-09-19T07:23:34.378 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.429+0000 7f1d02200640 1 -- 172.21.15.159:0/1097006010 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cfc07c2a0 con 0x7f1cfc107440 2024-09-19T07:23:34.378 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.429+0000 7f1d02200640 1 -- 172.21.15.159:0/1097006010 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cfc07c7e0 con 0x7f1cfc107440 2024-09-19T07:23:34.378 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.430+0000 7f1cf9600640 1 -- 172.21.15.159:0/1097006010 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f1cf003a7a0 con 0x7f1cfc107440 2024-09-19T07:23:34.378 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.430+0000 7f1d02200640 1 -- 172.21.15.159:0/1097006010 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cfc1078c0 con 0x7f1cfc107440 2024-09-19T07:23:34.378 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.431+0000 7f1cf9600640 1 --2- 172.21.15.159:0/1097006010 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1ce4039260 0x7f1ce403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:34.378 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.431+0000 7f1cf9600640 1 -- 172.21.15.159:0/1097006010 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1cf006c7d0 con 0x7f1cfc107440 2024-09-19T07:23:34.378 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.431+0000 7f1cfbe00640 1 --2- 172.21.15.159:0/1097006010 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1ce4039260 0x7f1ce403b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:34.378 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.432+0000 7f1cfbe00640 1 --2- 172.21.15.159:0/1097006010 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1ce4039260 0x7f1ce403b720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80098c0 tx=0x7f1ce8006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:34.378 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.436+0000 7f1cf9600640 1 -- 172.21.15.159:0/1097006010 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1cf00371e0 con 0x7f1cfc107440 2024-09-19T07:23:34.378 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.622+0000 7f1d02200640 1 -- 172.21.15.159:0/1097006010 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f1cfc07d160 con 0x7f1ce4039260 2024-09-19T07:23:34.378 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.733+0000 7f1cf9600640 1 -- 172.21.15.159:0/1097006010 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f1cfc07d160 con 0x7f1ce4039260 2024-09-19T07:23:34.379 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.736+0000 7f1d02200640 1 -- 172.21.15.159:0/1097006010 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1ce4039260 msgr2=0x7f1ce403b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:34.379 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.736+0000 7f1d02200640 1 --2- 172.21.15.159:0/1097006010 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1ce4039260 0x7f1ce403b720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80098c0 tx=0x7f1ce8006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:34.379 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.736+0000 7f1d02200640 1 -- 172.21.15.159:0/1097006010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cfc107440 msgr2=0x7f1cfc07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:34.379 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.736+0000 7f1d02200640 1 --2- 172.21.15.159:0/1097006010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cfc107440 0x7f1cfc07db70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00097f0 tx=0x7f1cf00327b0 comp rx=0 tx=0).stop 2024-09-19T07:23:34.379 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.736+0000 7f1d02200640 1 -- 172.21.15.159:0/1097006010 shutdown_connections 2024-09-19T07:23:34.379 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.736+0000 7f1d02200640 1 --2- 172.21.15.159:0/1097006010 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1ce4039260 0x7f1ce403b720 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:34.379 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.736+0000 7f1d02200640 1 --2- 172.21.15.159:0/1097006010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cfc107440 0x7f1cfc07db70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:34.379 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.736+0000 7f1d02200640 1 -- 172.21.15.159:0/1097006010 >> 172.21.15.159:0/1097006010 conn(0x7f1cfc102c10 msgr2=0x7f1cfc103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:34.379 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.737+0000 7f1d02200640 1 -- 172.21.15.159:0/1097006010 shutdown_connections 2024-09-19T07:23:34.379 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:33.737+0000 7f1d02200640 1 -- 172.21.15.159:0/1097006010 wait complete. 2024-09-19T07:23:34.379 INFO:teuthology.orchestra.run.smithi159.stdout:Creating initial admin user... 2024-09-19T07:23:34.928 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:34 smithi159 ceph-mon[30472]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:23:34.928 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:34 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:34.928 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:34 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:34.929 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:34 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:36.674 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:36 smithi159 ceph-mon[30472]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:23:36.674 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:36 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:36.977 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$KGfGSnAwXWM1wAWM5wKkp../RGADYu/F4BgTz.NK7dxtOC0WbK16W", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726730615, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T07:23:36.977 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.958+0000 7f3bd1600640 1 Processor -- start 2024-09-19T07:23:36.977 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.959+0000 7f3bd1600640 1 -- start start 2024-09-19T07:23:36.977 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.959+0000 7f3bd1600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bcc109870 0x7f3bcc109c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:36.977 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.959+0000 7f3bd1600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc10a240 con 0x7f3bcc109870 2024-09-19T07:23:36.977 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.959+0000 7f3bcaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bcc109870 0x7f3bcc109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:36.978 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.959+0000 7f3bcaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bcc109870 0x7f3bcc109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50326/0 (socket says 172.21.15.159:50326) 2024-09-19T07:23:36.978 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.959+0000 7f3bcaa00640 1 -- 172.21.15.159:0/715795308 learned_addr learned my addr 172.21.15.159:0/715795308 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:36.978 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.960+0000 7f3bcaa00640 1 -- 172.21.15.159:0/715795308 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bcc10aa10 con 0x7f3bcc109870 2024-09-19T07:23:36.978 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.961+0000 7f3bcaa00640 1 --2- 172.21.15.159:0/715795308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bcc109870 0x7f3bcc109c70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009e80 tx=0x7f3bbc02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f65e31ed6ad37f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:36.978 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.961+0000 7f3bca000640 1 -- 172.21.15.159:0/715795308 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bbc036070 con 0x7f3bcc109870 2024-09-19T07:23:36.978 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.961+0000 7f3bca000640 1 -- 172.21.15.159:0/715795308 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bbc0318e0 con 0x7f3bcc109870 2024-09-19T07:23:36.978 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.961+0000 7f3bca000640 1 -- 172.21.15.159:0/715795308 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bbc007a00 con 0x7f3bcc109870 2024-09-19T07:23:36.978 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.962+0000 7f3bd1600640 1 -- 172.21.15.159:0/715795308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bcc109870 msgr2=0x7f3bcc109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:36.978 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.962+0000 7f3bd1600640 1 --2- 172.21.15.159:0/715795308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bcc109870 0x7f3bcc109c70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009e80 tx=0x7f3bbc02a7a0 comp rx=0 tx=0).stop 2024-09-19T07:23:36.978 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.962+0000 7f3bd1600640 1 -- 172.21.15.159:0/715795308 shutdown_connections 2024-09-19T07:23:36.978 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.962+0000 7f3bd1600640 1 --2- 172.21.15.159:0/715795308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bcc109870 0x7f3bcc109c70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:36.978 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.962+0000 7f3bd1600640 1 -- 172.21.15.159:0/715795308 >> 172.21.15.159:0/715795308 conn(0x7f3bcc07cf40 msgr2=0x7f3bcc07d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:36.978 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.962+0000 7f3bd1600640 1 -- 172.21.15.159:0/715795308 shutdown_connections 2024-09-19T07:23:36.979 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.963+0000 7f3bd1600640 1 -- 172.21.15.159:0/715795308 wait complete. 2024-09-19T07:23:36.979 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.963+0000 7f3bd1600640 1 Processor -- start 2024-09-19T07:23:36.979 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.964+0000 7f3bd1600640 1 -- start start 2024-09-19T07:23:36.979 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.964+0000 7f3bd1600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bcc19d9a0 0x7f3bcc19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:36.979 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.964+0000 7f3bd1600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc10ddc0 con 0x7f3bcc19d9a0 2024-09-19T07:23:36.979 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.964+0000 7f3bcaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bcc19d9a0 0x7f3bcc19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:36.979 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.964+0000 7f3bcaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bcc19d9a0 0x7f3bcc19ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50330/0 (socket says 172.21.15.159:50330) 2024-09-19T07:23:36.979 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.964+0000 7f3bcaa00640 1 -- 172.21.15.159:0/3755000870 learned_addr learned my addr 172.21.15.159:0/3755000870 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:36.979 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.964+0000 7f3bcaa00640 1 -- 172.21.15.159:0/3755000870 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bcc19e300 con 0x7f3bcc19d9a0 2024-09-19T07:23:36.979 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.965+0000 7f3bcaa00640 1 --2- 172.21.15.159:0/3755000870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bcc19d9a0 0x7f3bcc19ddc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc000f80 tx=0x7f3bbc031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:36.979 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.965+0000 7f3bc3e00640 1 -- 172.21.15.159:0/3755000870 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bbc041020 con 0x7f3bcc19d9a0 2024-09-19T07:23:36.979 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.965+0000 7f3bc3e00640 1 -- 172.21.15.159:0/3755000870 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bbc03c470 con 0x7f3bcc19d9a0 2024-09-19T07:23:36.980 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.965+0000 7f3bc3e00640 1 -- 172.21.15.159:0/3755000870 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bbc036040 con 0x7f3bcc19d9a0 2024-09-19T07:23:36.980 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.965+0000 7f3bd1600640 1 -- 172.21.15.159:0/3755000870 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bcc19e590 con 0x7f3bcc19d9a0 2024-09-19T07:23:36.980 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.965+0000 7f3bd1600640 1 -- 172.21.15.159:0/3755000870 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bcc1a10f0 con 0x7f3bcc19d9a0 2024-09-19T07:23:36.980 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.967+0000 7f3bd1600640 1 -- 172.21.15.159:0/3755000870 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bcc109c70 con 0x7f3bcc19d9a0 2024-09-19T07:23:36.980 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.967+0000 7f3bc3e00640 1 -- 172.21.15.159:0/3755000870 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f3bbc044020 con 0x7f3bcc19d9a0 2024-09-19T07:23:36.980 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.968+0000 7f3bc3e00640 1 --2- 172.21.15.159:0/3755000870 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3bb0039260 0x7f3bb003b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:36.981 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.968+0000 7f3bc3e00640 1 -- 172.21.15.159:0/3755000870 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3bbc06d4f0 con 0x7f3bcc19d9a0 2024-09-19T07:23:36.981 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.971+0000 7f3bc2000640 1 --2- 172.21.15.159:0/3755000870 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3bb0039260 0x7f3bb003b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:36.981 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.972+0000 7f3bc2000640 1 --2- 172.21.15.159:0/3755000870 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3bb0039260 0x7f3bb003b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40098c0 tx=0x7f3bb4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:36.981 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:34.976+0000 7f3bc3e00640 1 -- 172.21.15.159:0/3755000870 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3bbc04b020 con 0x7f3bcc19d9a0 2024-09-19T07:23:36.981 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:35.175+0000 7f3bd1600640 1 -- 172.21.15.159:0/3755000870 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f3bcc107c00 con 0x7f3bb0039260 2024-09-19T07:23:36.981 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:35.402+0000 7f3bc3e00640 1 -- 172.21.15.159:0/3755000870 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f3bcc107c00 con 0x7f3bb0039260 2024-09-19T07:23:36.981 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:35.405+0000 7f3bd1600640 1 -- 172.21.15.159:0/3755000870 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3bb0039260 msgr2=0x7f3bb003b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:36.981 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:35.405+0000 7f3bd1600640 1 --2- 172.21.15.159:0/3755000870 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3bb0039260 0x7f3bb003b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40098c0 tx=0x7f3bb4006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:36.981 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:35.405+0000 7f3bd1600640 1 -- 172.21.15.159:0/3755000870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bcc19d9a0 msgr2=0x7f3bcc19ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:36.981 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:35.405+0000 7f3bd1600640 1 --2- 172.21.15.159:0/3755000870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bcc19d9a0 0x7f3bcc19ddc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc000f80 tx=0x7f3bbc031770 comp rx=0 tx=0).stop 2024-09-19T07:23:36.981 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:35.405+0000 7f3bd1600640 1 -- 172.21.15.159:0/3755000870 shutdown_connections 2024-09-19T07:23:36.982 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:35.405+0000 7f3bd1600640 1 --2- 172.21.15.159:0/3755000870 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3bb0039260 0x7f3bb003b720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:36.982 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:35.405+0000 7f3bd1600640 1 --2- 172.21.15.159:0/3755000870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bcc19d9a0 0x7f3bcc19ddc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:36.982 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:35.405+0000 7f3bd1600640 1 -- 172.21.15.159:0/3755000870 >> 172.21.15.159:0/3755000870 conn(0x7f3bcc07cf40 msgr2=0x7f3bcc1074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:36.982 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:35.405+0000 7f3bd1600640 1 -- 172.21.15.159:0/3755000870 shutdown_connections 2024-09-19T07:23:36.982 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:35.405+0000 7f3bd1600640 1 -- 172.21.15.159:0/3755000870 wait complete. 2024-09-19T07:23:36.982 INFO:teuthology.orchestra.run.smithi159.stdout:Fetching dashboard port number... 2024-09-19T07:23:38.353 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T07:23:38.353 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.498+0000 7ff5c4800640 1 Processor -- start 2024-09-19T07:23:38.353 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.498+0000 7ff5c4800640 1 -- start start 2024-09-19T07:23:38.353 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.498+0000 7ff5c4800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5bc1098c0 0x7ff5bc109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:38.353 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.498+0000 7ff5c4800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc10a290 con 0x7ff5bc1098c0 2024-09-19T07:23:38.354 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.498+0000 7ff5c1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5bc1098c0 0x7ff5bc109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:38.354 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.499+0000 7ff5c1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5bc1098c0 0x7ff5bc109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50340/0 (socket says 172.21.15.159:50340) 2024-09-19T07:23:38.354 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.499+0000 7ff5c1e00640 1 -- 172.21.15.159:0/3937593479 learned_addr learned my addr 172.21.15.159:0/3937593479 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:38.354 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.499+0000 7ff5c1e00640 1 -- 172.21.15.159:0/3937593479 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc10aa70 con 0x7ff5bc1098c0 2024-09-19T07:23:38.354 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.500+0000 7ff5c1e00640 1 --2- 172.21.15.159:0/3937593479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5bc1098c0 0x7ff5bc109cc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009820 tx=0x7ff5b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2d4e2cf21c714c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:38.354 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.500+0000 7ff5c0a00640 1 -- 172.21.15.159:0/3937593479 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5b0037070 con 0x7ff5bc1098c0 2024-09-19T07:23:38.354 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.500+0000 7ff5c0a00640 1 -- 172.21.15.159:0/3937593479 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b0032920 con 0x7ff5bc1098c0 2024-09-19T07:23:38.354 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.500+0000 7ff5c0a00640 1 -- 172.21.15.159:0/3937593479 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5b0030810 con 0x7ff5bc1098c0 2024-09-19T07:23:38.354 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.501+0000 7ff5c4800640 1 -- 172.21.15.159:0/3937593479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5bc1098c0 msgr2=0x7ff5bc109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:38.354 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.501+0000 7ff5c4800640 1 --2- 172.21.15.159:0/3937593479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5bc1098c0 0x7ff5bc109cc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009820 tx=0x7ff5b002a270 comp rx=0 tx=0).stop 2024-09-19T07:23:38.354 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.501+0000 7ff5c4800640 1 -- 172.21.15.159:0/3937593479 shutdown_connections 2024-09-19T07:23:38.354 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.501+0000 7ff5c4800640 1 --2- 172.21.15.159:0/3937593479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5bc1098c0 0x7ff5bc109cc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:38.354 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.501+0000 7ff5c4800640 1 -- 172.21.15.159:0/3937593479 >> 172.21.15.159:0/3937593479 conn(0x7ff5bc07cf40 msgr2=0x7ff5bc07d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:38.355 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.501+0000 7ff5c4800640 1 -- 172.21.15.159:0/3937593479 shutdown_connections 2024-09-19T07:23:38.355 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.501+0000 7ff5c4800640 1 -- 172.21.15.159:0/3937593479 wait complete. 2024-09-19T07:23:38.355 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.502+0000 7ff5c4800640 1 Processor -- start 2024-09-19T07:23:38.355 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.502+0000 7ff5c4800640 1 -- start start 2024-09-19T07:23:38.355 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.502+0000 7ff5c4800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5bc1098c0 0x7ff5bc19d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:38.355 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.502+0000 7ff5c4800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc10de20 con 0x7ff5bc1098c0 2024-09-19T07:23:38.355 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.502+0000 7ff5c1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5bc1098c0 0x7ff5bc19d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:38.355 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.502+0000 7ff5c1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5bc1098c0 0x7ff5bc19d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50344/0 (socket says 172.21.15.159:50344) 2024-09-19T07:23:38.355 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.502+0000 7ff5c1e00640 1 -- 172.21.15.159:0/131422271 learned_addr learned my addr 172.21.15.159:0/131422271 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:38.355 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.503+0000 7ff5c1e00640 1 -- 172.21.15.159:0/131422271 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc19ddf0 con 0x7ff5bc1098c0 2024-09-19T07:23:38.355 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.503+0000 7ff5c1e00640 1 --2- 172.21.15.159:0/131422271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5bc1098c0 0x7ff5bc19d8b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0030560 tx=0x7ff5b0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:38.355 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.503+0000 7ff5b6a00640 1 -- 172.21.15.159:0/131422271 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5b0037040 con 0x7ff5bc1098c0 2024-09-19T07:23:38.355 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.503+0000 7ff5b6a00640 1 -- 172.21.15.159:0/131422271 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b0040070 con 0x7ff5bc1098c0 2024-09-19T07:23:38.355 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.503+0000 7ff5b6a00640 1 -- 172.21.15.159:0/131422271 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5b003a740 con 0x7ff5bc1098c0 2024-09-19T07:23:38.356 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.503+0000 7ff5c4800640 1 -- 172.21.15.159:0/131422271 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5bc19e080 con 0x7ff5bc1098c0 2024-09-19T07:23:38.356 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.503+0000 7ff5c4800640 1 -- 172.21.15.159:0/131422271 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5bc1a0d70 con 0x7ff5bc1098c0 2024-09-19T07:23:38.356 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.504+0000 7ff5b6a00640 1 -- 172.21.15.159:0/131422271 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7ff5b003bb90 con 0x7ff5bc1098c0 2024-09-19T07:23:38.356 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.504+0000 7ff5c4800640 1 -- 172.21.15.159:0/131422271 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5bc109d40 con 0x7ff5bc1098c0 2024-09-19T07:23:38.356 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.505+0000 7ff5b6a00640 1 --2- 172.21.15.159:0/131422271 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff5a4039300 0x7ff5a403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:38.356 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.505+0000 7ff5b6a00640 1 -- 172.21.15.159:0/131422271 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff5b006cb00 con 0x7ff5bc1098c0 2024-09-19T07:23:38.356 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.505+0000 7ff5c1400640 1 --2- 172.21.15.159:0/131422271 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff5a4039300 0x7ff5a403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:38.356 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.506+0000 7ff5c1400640 1 --2- 172.21.15.159:0/131422271 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff5a4039300 0x7ff5a403b7c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0098c0 tx=0x7ff5ac006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:38.356 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.510+0000 7ff5b6a00640 1 -- 172.21.15.159:0/131422271 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff5b006ee20 con 0x7ff5bc1098c0 2024-09-19T07:23:38.356 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.687+0000 7ff5c4800640 1 -- 172.21.15.159:0/131422271 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7ff5bc1a0f40 con 0x7ff5bc1098c0 2024-09-19T07:23:38.356 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.688+0000 7ff5b6a00640 1 -- 172.21.15.159:0/131422271 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7ff5b0042960 con 0x7ff5bc1098c0 2024-09-19T07:23:38.356 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.690+0000 7ff5c4800640 1 -- 172.21.15.159:0/131422271 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff5a4039300 msgr2=0x7ff5a403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:38.356 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.690+0000 7ff5c4800640 1 --2- 172.21.15.159:0/131422271 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff5a4039300 0x7ff5a403b7c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0098c0 tx=0x7ff5ac006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:38.357 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.690+0000 7ff5c4800640 1 -- 172.21.15.159:0/131422271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5bc1098c0 msgr2=0x7ff5bc19d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:38.357 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.690+0000 7ff5c4800640 1 --2- 172.21.15.159:0/131422271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5bc1098c0 0x7ff5bc19d8b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0030560 tx=0x7ff5b0032670 comp rx=0 tx=0).stop 2024-09-19T07:23:38.357 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.690+0000 7ff5c4800640 1 -- 172.21.15.159:0/131422271 shutdown_connections 2024-09-19T07:23:38.357 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.690+0000 7ff5c4800640 1 --2- 172.21.15.159:0/131422271 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff5a4039300 0x7ff5a403b7c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:38.357 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.690+0000 7ff5c4800640 1 --2- 172.21.15.159:0/131422271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5bc1098c0 0x7ff5bc19d8b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:38.357 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.690+0000 7ff5c4800640 1 -- 172.21.15.159:0/131422271 >> 172.21.15.159:0/131422271 conn(0x7ff5bc07cf40 msgr2=0x7ff5bc107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:38.357 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.690+0000 7ff5c4800640 1 -- 172.21.15.159:0/131422271 shutdown_connections 2024-09-19T07:23:38.357 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:37.690+0000 7ff5c4800640 1 -- 172.21.15.159:0/131422271 wait complete. 2024-09-19T07:23:38.369 INFO:teuthology.orchestra.run.smithi159.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T07:23:38.369 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stdout disabled 2024-09-19T07:23:38.382 INFO:teuthology.orchestra.run.smithi159.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T07:23:38.382 INFO:teuthology.orchestra.run.smithi159.stdout:systemctl: stdout inactive 2024-09-19T07:23:38.382 INFO:teuthology.orchestra.run.smithi159.stdout:firewalld.service is not enabled 2024-09-19T07:23:38.382 INFO:teuthology.orchestra.run.smithi159.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T07:23:38.384 INFO:teuthology.orchestra.run.smithi159.stdout:Ceph Dashboard is now available at: 2024-09-19T07:23:38.384 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:23:38.384 INFO:teuthology.orchestra.run.smithi159.stdout: URL: https://smithi159.front.sepia.ceph.com:8443/ 2024-09-19T07:23:38.384 INFO:teuthology.orchestra.run.smithi159.stdout: User: admin 2024-09-19T07:23:38.385 INFO:teuthology.orchestra.run.smithi159.stdout: Password: bk1i4mx3tx 2024-09-19T07:23:38.385 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:23:38.385 INFO:teuthology.orchestra.run.smithi159.stdout:Saving cluster configuration to /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config directory 2024-09-19T07:23:38.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:38 smithi159 ceph-mon[30472]: mgrmap e13: x(active, since 6s) 2024-09-19T07:23:38.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:38 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/131422271' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T07:23:39.812 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.896+0000 7f2d61200640 1 Processor -- start 2024-09-19T07:23:39.812 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.896+0000 7f2d61200640 1 -- start start 2024-09-19T07:23:39.812 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.897+0000 7f2d61200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d5c107ad0 0x7f2d5c107ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:39.812 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.897+0000 7f2d61200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c1084a0 con 0x7f2d5c107ad0 2024-09-19T07:23:39.812 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.897+0000 7f2d59e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d5c107ad0 0x7f2d5c107ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:39.812 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.897+0000 7f2d59e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d5c107ad0 0x7f2d5c107ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50358/0 (socket says 172.21.15.159:50358) 2024-09-19T07:23:39.812 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.897+0000 7f2d59e00640 1 -- 172.21.15.159:0/4000529492 learned_addr learned my addr 172.21.15.159:0/4000529492 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:39.812 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.897+0000 7f2d59e00640 1 -- 172.21.15.159:0/4000529492 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5c108620 con 0x7f2d5c107ad0 2024-09-19T07:23:39.812 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.898+0000 7f2d59e00640 1 --2- 172.21.15.159:0/4000529492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d5c107ad0 0x7f2d5c107ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009820 tx=0x7f2d4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93efd73a7eb1f46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:39.813 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.899+0000 7f2d58a00640 1 -- 172.21.15.159:0/4000529492 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d4c037070 con 0x7f2d5c107ad0 2024-09-19T07:23:39.813 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.899+0000 7f2d58a00640 1 -- 172.21.15.159:0/4000529492 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d4c032920 con 0x7f2d5c107ad0 2024-09-19T07:23:39.813 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.899+0000 7f2d58a00640 1 -- 172.21.15.159:0/4000529492 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d4c030810 con 0x7f2d5c107ad0 2024-09-19T07:23:39.813 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.899+0000 7f2d61200640 1 -- 172.21.15.159:0/4000529492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d5c107ad0 msgr2=0x7f2d5c107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:39.813 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.899+0000 7f2d61200640 1 --2- 172.21.15.159:0/4000529492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d5c107ad0 0x7f2d5c107ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009820 tx=0x7f2d4c02a270 comp rx=0 tx=0).stop 2024-09-19T07:23:39.813 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.899+0000 7f2d61200640 1 -- 172.21.15.159:0/4000529492 shutdown_connections 2024-09-19T07:23:39.813 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.899+0000 7f2d61200640 1 --2- 172.21.15.159:0/4000529492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d5c107ad0 0x7f2d5c107ed0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:39.813 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.899+0000 7f2d61200640 1 -- 172.21.15.159:0/4000529492 >> 172.21.15.159:0/4000529492 conn(0x7f2d5c101470 msgr2=0x7f2d5c103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:39.813 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.899+0000 7f2d61200640 1 -- 172.21.15.159:0/4000529492 shutdown_connections 2024-09-19T07:23:39.813 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.899+0000 7f2d61200640 1 -- 172.21.15.159:0/4000529492 wait complete. 2024-09-19T07:23:39.813 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.900+0000 7f2d61200640 1 Processor -- start 2024-09-19T07:23:39.813 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.900+0000 7f2d61200640 1 -- start start 2024-09-19T07:23:39.813 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.901+0000 7f2d61200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d5c107ad0 0x7f2d5c19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:39.814 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.901+0000 7f2d61200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c1064f0 con 0x7f2d5c107ad0 2024-09-19T07:23:39.814 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.901+0000 7f2d59e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d5c107ad0 0x7f2d5c19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:39.814 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.901+0000 7f2d59e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d5c107ad0 0x7f2d5c19d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50366/0 (socket says 172.21.15.159:50366) 2024-09-19T07:23:39.814 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.901+0000 7f2d59e00640 1 -- 172.21.15.159:0/3144249669 learned_addr learned my addr 172.21.15.159:0/3144249669 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:39.814 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.901+0000 7f2d59e00640 1 -- 172.21.15.159:0/3144249669 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5c19dcf0 con 0x7f2d5c107ad0 2024-09-19T07:23:39.814 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.902+0000 7f2d59e00640 1 --2- 172.21.15.159:0/3144249669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d5c107ad0 0x7f2d5c19d7b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0097f0 tx=0x7f2d4c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:39.814 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.902+0000 7f2d52a00640 1 -- 172.21.15.159:0/3144249669 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d4c037040 con 0x7f2d5c107ad0 2024-09-19T07:23:39.814 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.902+0000 7f2d52a00640 1 -- 172.21.15.159:0/3144249669 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d4c030cc0 con 0x7f2d5c107ad0 2024-09-19T07:23:39.814 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.902+0000 7f2d61200640 1 -- 172.21.15.159:0/3144249669 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d5c19df80 con 0x7f2d5c107ad0 2024-09-19T07:23:39.814 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.902+0000 7f2d52a00640 1 -- 172.21.15.159:0/3144249669 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d4c03a6b0 con 0x7f2d5c107ad0 2024-09-19T07:23:39.814 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.902+0000 7f2d61200640 1 -- 172.21.15.159:0/3144249669 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d5c1a0c70 con 0x7f2d5c107ad0 2024-09-19T07:23:39.814 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.903+0000 7f2d61200640 1 -- 172.21.15.159:0/3144249669 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d5c107f50 con 0x7f2d5c107ad0 2024-09-19T07:23:39.814 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.903+0000 7f2d52a00640 1 -- 172.21.15.159:0/3144249669 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f2d4c03b3f0 con 0x7f2d5c107ad0 2024-09-19T07:23:39.814 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.903+0000 7f2d52a00640 1 --2- 172.21.15.159:0/3144249669 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2d40039300 0x7f2d4003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:39.815 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.904+0000 7f2d52a00640 1 -- 172.21.15.159:0/3144249669 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2d4c06ca10 con 0x7f2d5c107ad0 2024-09-19T07:23:39.815 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.904+0000 7f2d59400640 1 --2- 172.21.15.159:0/3144249669 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2d40039300 0x7f2d4003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:39.815 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.904+0000 7f2d59400640 1 --2- 172.21.15.159:0/3144249669 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2d40039300 0x7f2d4003b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2d480098c0 tx=0x7f2d48006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:39.815 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:38.911+0000 7f2d52a00640 1 -- 172.21.15.159:0/3144249669 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2d4c031e80 con 0x7f2d5c107ad0 2024-09-19T07:23:39.815 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:39.173+0000 7f2d61200640 1 -- 172.21.15.159:0/3144249669 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f2d5c196c20 con 0x7f2d5c107ad0 2024-09-19T07:23:39.815 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:39.175+0000 7f2d52a00640 1 -- 172.21.15.159:0/3144249669 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f2d4c02fd60 con 0x7f2d5c107ad0 2024-09-19T07:23:39.815 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T07:23:39.815 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:39.178+0000 7f2d61200640 1 -- 172.21.15.159:0/3144249669 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2d40039300 msgr2=0x7f2d4003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:39.815 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:39.178+0000 7f2d61200640 1 --2- 172.21.15.159:0/3144249669 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2d40039300 0x7f2d4003b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2d480098c0 tx=0x7f2d48006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:39.815 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:39.178+0000 7f2d61200640 1 -- 172.21.15.159:0/3144249669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d5c107ad0 msgr2=0x7f2d5c19d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:39.815 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:39.178+0000 7f2d61200640 1 --2- 172.21.15.159:0/3144249669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d5c107ad0 0x7f2d5c19d7b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0097f0 tx=0x7f2d4c0327b0 comp rx=0 tx=0).stop 2024-09-19T07:23:39.815 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:39.178+0000 7f2d61200640 1 -- 172.21.15.159:0/3144249669 shutdown_connections 2024-09-19T07:23:39.815 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:39.178+0000 7f2d61200640 1 --2- 172.21.15.159:0/3144249669 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2d40039300 0x7f2d4003b7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:39.816 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:39.178+0000 7f2d61200640 1 --2- 172.21.15.159:0/3144249669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d5c107ad0 0x7f2d5c19d7b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:39.816 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:39.178+0000 7f2d61200640 1 -- 172.21.15.159:0/3144249669 >> 172.21.15.159:0/3144249669 conn(0x7f2d5c101470 msgr2=0x7f2d5c193d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:39.816 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:39.178+0000 7f2d61200640 1 -- 172.21.15.159:0/3144249669 shutdown_connections 2024-09-19T07:23:39.816 INFO:teuthology.orchestra.run.smithi159.stdout:/usr/bin/ceph: stderr 2024-09-19T07:23:39.178+0000 7f2d61200640 1 -- 172.21.15.159:0/3144249669 wait complete. 2024-09-19T07:23:39.816 INFO:teuthology.orchestra.run.smithi159.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T07:23:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:23:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:23:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:23:39.816 INFO:teuthology.orchestra.run.smithi159.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T07:23:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:23:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T07:23:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:23:39.816 INFO:teuthology.orchestra.run.smithi159.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T07:23:39.816 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:23:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: ceph telemetry on 2024-09-19T07:23:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:23:39.817 INFO:teuthology.orchestra.run.smithi159.stdout:For more information see: 2024-09-19T07:23:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:23:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T07:23:39.817 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:23:39.817 INFO:teuthology.orchestra.run.smithi159.stdout:Bootstrap complete. 2024-09-19T07:23:39.870 INFO:tasks.cephadm:Fetching config... 2024-09-19T07:23:39.870 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:23:39.870 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T07:23:39.906 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T07:23:39.907 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:23:39.907 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T07:23:39.984 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T07:23:39.985 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:23:39.985 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-19T07:23:40.084 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T07:23:40.084 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:23:40.084 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T07:23:40.159 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T07:23:40.159 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD40dGyPTYYo29uJxWEgmfUplFmtY/xf37wV6O9MUpHiatxI+Xq4WCYJxQT9Nf7jB0ijS/qxKZeCG7xng6vAsSAEnxwSLxTPNOYI0bkfLZvhpplD5dOg/tlrycCkXsotbfeosP2k6atNnCIlQV5E+FSQTEuqMfV4IoFerFyXULbJUP7rjPfIyxCmNCPKzlTipF6AHCvy2CI1iu29SRz4WYD+Cmlo/3bXmXdid0cPHPHsgjg4WNorQE3fBfxMUZjTemcjPVxElNRrSVI8xp4NIG7mx2Gy1B8J2meyGLSTtR/qLwHWB56lhKM/+9ZIjdYR3AA1U7jolLD/4CEEYCBG6IRrNGo74rRuEHTCLxK3Z8vPfNEy1C915LPt9vw0zNl4Z08N7zNy82vrvaScR6w9lornEKd9jrDnRmXfafxsOinDNZOf53qAa2mU1tvcZM4kgQbAG31WAIdl539kvmOlcwXaaORGdVn7eKVPC2lg1+0gJYlcNhXy9JcW4meMAO0/U0= ceph-c9d18048-7657-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T07:23:40.280 INFO:teuthology.orchestra.run.smithi159.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD40dGyPTYYo29uJxWEgmfUplFmtY/xf37wV6O9MUpHiatxI+Xq4WCYJxQT9Nf7jB0ijS/qxKZeCG7xng6vAsSAEnxwSLxTPNOYI0bkfLZvhpplD5dOg/tlrycCkXsotbfeosP2k6atNnCIlQV5E+FSQTEuqMfV4IoFerFyXULbJUP7rjPfIyxCmNCPKzlTipF6AHCvy2CI1iu29SRz4WYD+Cmlo/3bXmXdid0cPHPHsgjg4WNorQE3fBfxMUZjTemcjPVxElNRrSVI8xp4NIG7mx2Gy1B8J2meyGLSTtR/qLwHWB56lhKM/+9ZIjdYR3AA1U7jolLD/4CEEYCBG6IRrNGo74rRuEHTCLxK3Z8vPfNEy1C915LPt9vw0zNl4Z08N7zNy82vrvaScR6w9lornEKd9jrDnRmXfafxsOinDNZOf53qAa2mU1tvcZM4kgQbAG31WAIdl539kvmOlcwXaaORGdVn7eKVPC2lg1+0gJYlcNhXy9JcW4meMAO0/U0= ceph-c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:23:40.305 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD40dGyPTYYo29uJxWEgmfUplFmtY/xf37wV6O9MUpHiatxI+Xq4WCYJxQT9Nf7jB0ijS/qxKZeCG7xng6vAsSAEnxwSLxTPNOYI0bkfLZvhpplD5dOg/tlrycCkXsotbfeosP2k6atNnCIlQV5E+FSQTEuqMfV4IoFerFyXULbJUP7rjPfIyxCmNCPKzlTipF6AHCvy2CI1iu29SRz4WYD+Cmlo/3bXmXdid0cPHPHsgjg4WNorQE3fBfxMUZjTemcjPVxElNRrSVI8xp4NIG7mx2Gy1B8J2meyGLSTtR/qLwHWB56lhKM/+9ZIjdYR3AA1U7jolLD/4CEEYCBG6IRrNGo74rRuEHTCLxK3Z8vPfNEy1C915LPt9vw0zNl4Z08N7zNy82vrvaScR6w9lornEKd9jrDnRmXfafxsOinDNZOf53qAa2mU1tvcZM4kgQbAG31WAIdl539kvmOlcwXaaORGdVn7eKVPC2lg1+0gJYlcNhXy9JcW4meMAO0/U0= ceph-c9d18048-7657-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T07:23:40.351 INFO:teuthology.orchestra.run.smithi189.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD40dGyPTYYo29uJxWEgmfUplFmtY/xf37wV6O9MUpHiatxI+Xq4WCYJxQT9Nf7jB0ijS/qxKZeCG7xng6vAsSAEnxwSLxTPNOYI0bkfLZvhpplD5dOg/tlrycCkXsotbfeosP2k6atNnCIlQV5E+FSQTEuqMfV4IoFerFyXULbJUP7rjPfIyxCmNCPKzlTipF6AHCvy2CI1iu29SRz4WYD+Cmlo/3bXmXdid0cPHPHsgjg4WNorQE3fBfxMUZjTemcjPVxElNRrSVI8xp4NIG7mx2Gy1B8J2meyGLSTtR/qLwHWB56lhKM/+9ZIjdYR3AA1U7jolLD/4CEEYCBG6IRrNGo74rRuEHTCLxK3Z8vPfNEy1C915LPt9vw0zNl4Z08N7zNy82vrvaScR6w9lornEKd9jrDnRmXfafxsOinDNZOf53qAa2mU1tvcZM4kgQbAG31WAIdl539kvmOlcwXaaORGdVn7eKVPC2lg1+0gJYlcNhXy9JcW4meMAO0/U0= ceph-c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:23:40.363 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD40dGyPTYYo29uJxWEgmfUplFmtY/xf37wV6O9MUpHiatxI+Xq4WCYJxQT9Nf7jB0ijS/qxKZeCG7xng6vAsSAEnxwSLxTPNOYI0bkfLZvhpplD5dOg/tlrycCkXsotbfeosP2k6atNnCIlQV5E+FSQTEuqMfV4IoFerFyXULbJUP7rjPfIyxCmNCPKzlTipF6AHCvy2CI1iu29SRz4WYD+Cmlo/3bXmXdid0cPHPHsgjg4WNorQE3fBfxMUZjTemcjPVxElNRrSVI8xp4NIG7mx2Gy1B8J2meyGLSTtR/qLwHWB56lhKM/+9ZIjdYR3AA1U7jolLD/4CEEYCBG6IRrNGo74rRuEHTCLxK3Z8vPfNEy1C915LPt9vw0zNl4Z08N7zNy82vrvaScR6w9lornEKd9jrDnRmXfafxsOinDNZOf53qAa2mU1tvcZM4kgQbAG31WAIdl539kvmOlcwXaaORGdVn7eKVPC2lg1+0gJYlcNhXy9JcW4meMAO0/U0= ceph-c9d18048-7657-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T07:23:40.369 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:40 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3144249669' entity='client.admin' 2024-09-19T07:23:40.408 INFO:teuthology.orchestra.run.smithi203.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD40dGyPTYYo29uJxWEgmfUplFmtY/xf37wV6O9MUpHiatxI+Xq4WCYJxQT9Nf7jB0ijS/qxKZeCG7xng6vAsSAEnxwSLxTPNOYI0bkfLZvhpplD5dOg/tlrycCkXsotbfeosP2k6atNnCIlQV5E+FSQTEuqMfV4IoFerFyXULbJUP7rjPfIyxCmNCPKzlTipF6AHCvy2CI1iu29SRz4WYD+Cmlo/3bXmXdid0cPHPHsgjg4WNorQE3fBfxMUZjTemcjPVxElNRrSVI8xp4NIG7mx2Gy1B8J2meyGLSTtR/qLwHWB56lhKM/+9ZIjdYR3AA1U7jolLD/4CEEYCBG6IRrNGo74rRuEHTCLxK3Z8vPfNEy1C915LPt9vw0zNl4Z08N7zNy82vrvaScR6w9lornEKd9jrDnRmXfafxsOinDNZOf53qAa2mU1tvcZM4kgQbAG31WAIdl539kvmOlcwXaaORGdVn7eKVPC2lg1+0gJYlcNhXy9JcW4meMAO0/U0= ceph-c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:23:40.421 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T07:23:40.790 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:23:41.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.461+0000 7fb207800640 1 -- 172.21.15.159:0/1213377931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb200103820 msgr2=0x7fb200103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:41.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.461+0000 7fb207800640 1 --2- 172.21.15.159:0/1213377931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb200103820 0x7fb200103c20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0009880 tx=0x7fb1f002a5d0 comp rx=0 tx=0).stop 2024-09-19T07:23:41.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.461+0000 7fb207800640 1 -- 172.21.15.159:0/1213377931 shutdown_connections 2024-09-19T07:23:41.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.461+0000 7fb207800640 1 --2- 172.21.15.159:0/1213377931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb200103820 0x7fb200103c20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:41.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.461+0000 7fb207800640 1 -- 172.21.15.159:0/1213377931 >> 172.21.15.159:0/1213377931 conn(0x7fb2000feff0 msgr2=0x7fb200101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:41.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.461+0000 7fb207800640 1 -- 172.21.15.159:0/1213377931 shutdown_connections 2024-09-19T07:23:41.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.461+0000 7fb207800640 1 -- 172.21.15.159:0/1213377931 wait complete. 2024-09-19T07:23:41.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.462+0000 7fb207800640 1 Processor -- start 2024-09-19T07:23:41.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.462+0000 7fb207800640 1 -- start start 2024-09-19T07:23:41.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.463+0000 7fb207800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb200103820 0x7fb200198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:41.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.463+0000 7fb207800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb200109930 con 0x7fb200103820 2024-09-19T07:23:41.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.463+0000 7fb204e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb200103820 0x7fb200198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:41.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.463+0000 7fb204e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb200103820 0x7fb200198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50376/0 (socket says 172.21.15.159:50376) 2024-09-19T07:23:41.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.463+0000 7fb204e00640 1 -- 172.21.15.159:0/1973802226 learned_addr learned my addr 172.21.15.159:0/1973802226 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:41.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.463+0000 7fb204e00640 1 -- 172.21.15.159:0/1973802226 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2001994c0 con 0x7fb200103820 2024-09-19T07:23:41.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.464+0000 7fb204e00640 1 --2- 172.21.15.159:0/1973802226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb200103820 0x7fb200198f80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0004f60 tx=0x7fb1f00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:41.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.464+0000 7fb1fe000640 1 -- 172.21.15.159:0/1973802226 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1f003f070 con 0x7fb200103820 2024-09-19T07:23:41.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.464+0000 7fb1fe000640 1 -- 172.21.15.159:0/1973802226 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1f0004cc0 con 0x7fb200103820 2024-09-19T07:23:41.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.464+0000 7fb1fe000640 1 -- 172.21.15.159:0/1973802226 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1f0035030 con 0x7fb200103820 2024-09-19T07:23:41.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.464+0000 7fb207800640 1 -- 172.21.15.159:0/1973802226 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb200199750 con 0x7fb200103820 2024-09-19T07:23:41.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.464+0000 7fb207800640 1 -- 172.21.15.159:0/1973802226 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb20019c440 con 0x7fb200103820 2024-09-19T07:23:41.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.465+0000 7fb1fe000640 1 -- 172.21.15.159:0/1973802226 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fb1f0042070 con 0x7fb200103820 2024-09-19T07:23:41.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.465+0000 7fb207800640 1 -- 172.21.15.159:0/1973802226 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb20010a0b0 con 0x7fb200103820 2024-09-19T07:23:41.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.466+0000 7fb1fe000640 1 --2- 172.21.15.159:0/1973802226 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb1e4039300 0x7fb1e403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:41.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.466+0000 7fb1fe000640 1 -- 172.21.15.159:0/1973802226 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb1f006d0c0 con 0x7fb200103820 2024-09-19T07:23:41.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.466+0000 7fb1f7e00640 1 --2- 172.21.15.159:0/1973802226 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb1e4039300 0x7fb1e403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:41.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.467+0000 7fb1f7e00640 1 --2- 172.21.15.159:0/1973802226 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb1e4039300 0x7fb1e403b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80098c0 tx=0x7fb1e8006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:41.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.470+0000 7fb1fe000640 1 -- 172.21.15.159:0/1973802226 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb1f0040260 con 0x7fb200103820 2024-09-19T07:23:41.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.650+0000 7fb207800640 1 -- 172.21.15.159:0/1973802226 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fb20002d830 con 0x7fb200103820 2024-09-19T07:23:41.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.653+0000 7fb1fe000640 1 -- 172.21.15.159:0/1973802226 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fb1f0003dd0 con 0x7fb200103820 2024-09-19T07:23:41.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.655+0000 7fb207800640 1 -- 172.21.15.159:0/1973802226 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb1e4039300 msgr2=0x7fb1e403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:41.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.655+0000 7fb207800640 1 --2- 172.21.15.159:0/1973802226 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb1e4039300 0x7fb1e403b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80098c0 tx=0x7fb1e8006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:41.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.655+0000 7fb207800640 1 -- 172.21.15.159:0/1973802226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb200103820 msgr2=0x7fb200198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:41.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.655+0000 7fb207800640 1 --2- 172.21.15.159:0/1973802226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb200103820 0x7fb200198f80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0004f60 tx=0x7fb1f00048f0 comp rx=0 tx=0).stop 2024-09-19T07:23:41.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.655+0000 7fb207800640 1 -- 172.21.15.159:0/1973802226 shutdown_connections 2024-09-19T07:23:41.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.655+0000 7fb207800640 1 --2- 172.21.15.159:0/1973802226 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb1e4039300 0x7fb1e403b7c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:41.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.655+0000 7fb207800640 1 --2- 172.21.15.159:0/1973802226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb200103820 0x7fb200198f80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:41.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.655+0000 7fb207800640 1 -- 172.21.15.159:0/1973802226 >> 172.21.15.159:0/1973802226 conn(0x7fb2000feff0 msgr2=0x7fb2000ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:41.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.655+0000 7fb207800640 1 -- 172.21.15.159:0/1973802226 shutdown_connections 2024-09-19T07:23:41.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:41.655+0000 7fb207800640 1 -- 172.21.15.159:0/1973802226 wait complete. 2024-09-19T07:23:42.407 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T07:23:42.407 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T07:23:42.957 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:23:43.082 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:42 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1973802226' entity='client.admin' 2024-09-19T07:23:43.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:43 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:43.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:43 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:43.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:43 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:23:43.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:43 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:43.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:43 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:23:43.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:43 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:44.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.037+0000 7f64ffc00640 1 -- 172.21.15.159:0/429577065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f8101620 msgr2=0x7f64f8101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:44.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.037+0000 7f64ffc00640 1 --2- 172.21.15.159:0/429577065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f8101620 0x7f64f8101a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009880 tx=0x7f64ec02a5d0 comp rx=0 tx=0).stop 2024-09-19T07:23:44.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.037+0000 7f64ffc00640 1 -- 172.21.15.159:0/429577065 shutdown_connections 2024-09-19T07:23:44.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.037+0000 7f64ffc00640 1 --2- 172.21.15.159:0/429577065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f8101620 0x7f64f8101a20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:44.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.037+0000 7f64ffc00640 1 -- 172.21.15.159:0/429577065 >> 172.21.15.159:0/429577065 conn(0x7f64f80fcdb0 msgr2=0x7f64f80ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:44.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.037+0000 7f64ffc00640 1 -- 172.21.15.159:0/429577065 shutdown_connections 2024-09-19T07:23:44.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.037+0000 7f64ffc00640 1 -- 172.21.15.159:0/429577065 wait complete. 2024-09-19T07:23:44.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.039+0000 7f64ffc00640 1 Processor -- start 2024-09-19T07:23:44.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.039+0000 7f64ffc00640 1 -- start start 2024-09-19T07:23:44.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.039+0000 7f64ffc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f8101620 0x7f64f8198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:44.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.039+0000 7f64ffc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64f8107730 con 0x7f64f8101620 2024-09-19T07:23:44.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.040+0000 7f64fd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f8101620 0x7f64f8198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:44.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.040+0000 7f64fd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f8101620 0x7f64f8198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41058/0 (socket says 172.21.15.159:41058) 2024-09-19T07:23:44.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.040+0000 7f64fd200640 1 -- 172.21.15.159:0/157388657 learned_addr learned my addr 172.21.15.159:0/157388657 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:44.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.040+0000 7f64fd200640 1 -- 172.21.15.159:0/157388657 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f81994d0 con 0x7f64f8101620 2024-09-19T07:23:44.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.040+0000 7f64fd200640 1 --2- 172.21.15.159:0/157388657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f8101620 0x7f64f8198f90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f64ec004f60 tx=0x7f64ec0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:44.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.041+0000 7f64f5600640 1 -- 172.21.15.159:0/157388657 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64ec03f070 con 0x7f64f8101620 2024-09-19T07:23:44.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.041+0000 7f64f5600640 1 -- 172.21.15.159:0/157388657 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64ec004d40 con 0x7f64f8101620 2024-09-19T07:23:44.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.041+0000 7f64f5600640 1 -- 172.21.15.159:0/157388657 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64ec035030 con 0x7f64f8101620 2024-09-19T07:23:44.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.041+0000 7f64ffc00640 1 -- 172.21.15.159:0/157388657 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f8199760 con 0x7f64f8101620 2024-09-19T07:23:44.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.041+0000 7f64ffc00640 1 -- 172.21.15.159:0/157388657 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f819c450 con 0x7f64f8101620 2024-09-19T07:23:44.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.042+0000 7f64ffc00640 1 -- 172.21.15.159:0/157388657 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f8107eb0 con 0x7f64f8101620 2024-09-19T07:23:44.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.042+0000 7f64f5600640 1 -- 172.21.15.159:0/157388657 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f64ec042070 con 0x7f64f8101620 2024-09-19T07:23:44.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.043+0000 7f64f5600640 1 --2- 172.21.15.159:0/157388657 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f64e0039300 0x7f64e003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:44.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.043+0000 7f64f5600640 1 -- 172.21.15.159:0/157388657 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f64ec06cff0 con 0x7f64f8101620 2024-09-19T07:23:44.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.043+0000 7f64f7e00640 1 --2- 172.21.15.159:0/157388657 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f64e0039300 0x7f64e003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:44.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.044+0000 7f64f7e00640 1 --2- 172.21.15.159:0/157388657 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f64e0039300 0x7f64e003b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f64e40098c0 tx=0x7f64e4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:44.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.048+0000 7f64f5600640 1 -- 172.21.15.159:0/157388657 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f64ec04a260 con 0x7f64f8101620 2024-09-19T07:23:44.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.237+0000 7f64ffc00640 1 -- 172.21.15.159:0/157388657 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f64f8062630 con 0x7f64e0039300 2024-09-19T07:23:44.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.241+0000 7f64f5600640 1 -- 172.21.15.159:0/157388657 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f64f8062630 con 0x7f64e0039300 2024-09-19T07:23:44.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.244+0000 7f64ffc00640 1 -- 172.21.15.159:0/157388657 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f64e0039300 msgr2=0x7f64e003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:44.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.244+0000 7f64ffc00640 1 --2- 172.21.15.159:0/157388657 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f64e0039300 0x7f64e003b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f64e40098c0 tx=0x7f64e4006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:44.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.244+0000 7f64ffc00640 1 -- 172.21.15.159:0/157388657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f8101620 msgr2=0x7f64f8198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:44.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.244+0000 7f64ffc00640 1 --2- 172.21.15.159:0/157388657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f8101620 0x7f64f8198f90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f64ec004f60 tx=0x7f64ec0048f0 comp rx=0 tx=0).stop 2024-09-19T07:23:44.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.244+0000 7f64ffc00640 1 -- 172.21.15.159:0/157388657 shutdown_connections 2024-09-19T07:23:44.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.244+0000 7f64ffc00640 1 --2- 172.21.15.159:0/157388657 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f64e0039300 0x7f64e003b7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:44.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.244+0000 7f64ffc00640 1 --2- 172.21.15.159:0/157388657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f8101620 0x7f64f8198f90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:44.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.244+0000 7f64ffc00640 1 -- 172.21.15.159:0/157388657 >> 172.21.15.159:0/157388657 conn(0x7f64f80fcdb0 msgr2=0x7f64f80ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:44.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.244+0000 7f64ffc00640 1 -- 172.21.15.159:0/157388657 shutdown_connections 2024-09-19T07:23:44.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:44.244+0000 7f64ffc00640 1 -- 172.21.15.159:0/157388657 wait complete. 2024-09-19T07:23:44.997 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi189 2024-09-19T07:23:44.997 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-19T07:23:44.997 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.conf 2024-09-19T07:23:45.016 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-19T07:23:45.016 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:23:45.074 INFO:tasks.cephadm:Adding host smithi189 to orchestrator... 2024-09-19T07:23:45.075 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch host add smithi189 2024-09-19T07:23:45.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:45 smithi159 ceph-mon[30472]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:23:45.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:45 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:45.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:45 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:23:45.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:45 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:23:45.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:45 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:23:45.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:45 smithi159 ceph-mon[30472]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-19T07:23:45.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:45 smithi159 ceph-mon[30472]: Updating smithi159:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:23:45.804 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:23:46.343 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:46 smithi159 ceph-mon[30472]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:23:46.343 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:46 smithi159 ceph-mon[30472]: Updating smithi159:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:23:46.343 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:46 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:46.343 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:46 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:46.343 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:46 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:46.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.456+0000 7f48b3e00640 1 -- 172.21.15.159:0/2467280432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48b41015e0 msgr2=0x7f48b41019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:46.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.456+0000 7f48b3e00640 1 --2- 172.21.15.159:0/2467280432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48b41015e0 0x7f48b41019c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009880 tx=0x7f48a402a5d0 comp rx=0 tx=0).stop 2024-09-19T07:23:46.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.457+0000 7f48b3e00640 1 -- 172.21.15.159:0/2467280432 shutdown_connections 2024-09-19T07:23:46.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.457+0000 7f48b3e00640 1 --2- 172.21.15.159:0/2467280432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48b41015e0 0x7f48b41019c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:46.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.457+0000 7f48b3e00640 1 -- 172.21.15.159:0/2467280432 >> 172.21.15.159:0/2467280432 conn(0x7f48b4078240 msgr2=0x7f48b40ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:46.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.457+0000 7f48b3e00640 1 -- 172.21.15.159:0/2467280432 shutdown_connections 2024-09-19T07:23:46.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.457+0000 7f48b3e00640 1 -- 172.21.15.159:0/2467280432 wait complete. 2024-09-19T07:23:46.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.458+0000 7f48b3e00640 1 Processor -- start 2024-09-19T07:23:46.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.458+0000 7f48b3e00640 1 -- start start 2024-09-19T07:23:46.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.458+0000 7f48b3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48b41015e0 0x7f48b40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:46.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.458+0000 7f48b3e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f48b410ba30 con 0x7f48b41015e0 2024-09-19T07:23:46.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.459+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48b41015e0 0x7f48b40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:46.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.459+0000 7f48b2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48b41015e0 0x7f48b40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41076/0 (socket says 172.21.15.159:41076) 2024-09-19T07:23:46.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.459+0000 7f48b2a00640 1 -- 172.21.15.159:0/4129315350 learned_addr learned my addr 172.21.15.159:0/4129315350 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:46.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.459+0000 7f48b2a00640 1 -- 172.21.15.159:0/4129315350 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b4076390 con 0x7f48b41015e0 2024-09-19T07:23:46.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.459+0000 7f48b2a00640 1 --2- 172.21.15.159:0/4129315350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48b41015e0 0x7f48b40744f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009230 tx=0x7f48a4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:46.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.460+0000 7f48ab400640 1 -- 172.21.15.159:0/4129315350 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48a403f070 con 0x7f48b41015e0 2024-09-19T07:23:46.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.460+0000 7f48ab400640 1 -- 172.21.15.159:0/4129315350 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48a403bc80 con 0x7f48b41015e0 2024-09-19T07:23:46.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.460+0000 7f48ab400640 1 -- 172.21.15.159:0/4129315350 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48a4035030 con 0x7f48b41015e0 2024-09-19T07:23:46.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.460+0000 7f48b3e00640 1 -- 172.21.15.159:0/4129315350 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b4074a30 con 0x7f48b41015e0 2024-09-19T07:23:46.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.460+0000 7f48b3e00640 1 -- 172.21.15.159:0/4129315350 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b4074ef0 con 0x7f48b41015e0 2024-09-19T07:23:46.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.461+0000 7f48ab400640 1 -- 172.21.15.159:0/4129315350 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f48a4042070 con 0x7f48b41015e0 2024-09-19T07:23:46.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.461+0000 7f48b3e00640 1 -- 172.21.15.159:0/4129315350 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48b4102ac0 con 0x7f48b41015e0 2024-09-19T07:23:46.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.461+0000 7f48ab400640 1 --2- 172.21.15.159:0/4129315350 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4898039300 0x7f489803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:46.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.461+0000 7f48ab400640 1 -- 172.21.15.159:0/4129315350 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f48a406cf50 con 0x7f48b41015e0 2024-09-19T07:23:46.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.462+0000 7f48b2000640 1 --2- 172.21.15.159:0/4129315350 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4898039300 0x7f489803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:46.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.462+0000 7f48b2000640 1 --2- 172.21.15.159:0/4129315350 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4898039300 0x7f489803b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f48a00098c0 tx=0x7f48a0006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:46.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.466+0000 7f48ab400640 1 -- 172.21.15.159:0/4129315350 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f48a40372d0 con 0x7f48b41015e0 2024-09-19T07:23:46.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:46.648+0000 7f48b3e00640 1 -- 172.21.15.159:0/4129315350 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}) -- 0x7f48b40759d0 con 0x7f4898039300 2024-09-19T07:23:47.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:47 smithi159 ceph-mon[30472]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:23:48.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:48 smithi159 ceph-mon[30472]: Deploying cephadm binary to smithi189 2024-09-19T07:23:49.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:49.841+0000 7f48ab400640 1 -- 172.21.15.159:0/4129315350 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f48b40759d0 con 0x7f4898039300 2024-09-19T07:23:49.841 INFO:teuthology.orchestra.run.smithi159.stdout:Added host 'smithi189' with addr '172.21.15.189' 2024-09-19T07:23:49.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:49.844+0000 7f48b3e00640 1 -- 172.21.15.159:0/4129315350 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4898039300 msgr2=0x7f489803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:49.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:49.844+0000 7f48b3e00640 1 --2- 172.21.15.159:0/4129315350 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4898039300 0x7f489803b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f48a00098c0 tx=0x7f48a0006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:49.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:49.844+0000 7f48b3e00640 1 -- 172.21.15.159:0/4129315350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48b41015e0 msgr2=0x7f48b40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:49.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:49.844+0000 7f48b3e00640 1 --2- 172.21.15.159:0/4129315350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48b41015e0 0x7f48b40744f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009230 tx=0x7f48a4003d40 comp rx=0 tx=0).stop 2024-09-19T07:23:49.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:49.844+0000 7f48b3e00640 1 -- 172.21.15.159:0/4129315350 shutdown_connections 2024-09-19T07:23:49.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:49.844+0000 7f48b3e00640 1 --2- 172.21.15.159:0/4129315350 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4898039300 0x7f489803b7c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:49.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:49.844+0000 7f48b3e00640 1 --2- 172.21.15.159:0/4129315350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48b41015e0 0x7f48b40744f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:49.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:49.844+0000 7f48b3e00640 1 -- 172.21.15.159:0/4129315350 >> 172.21.15.159:0/4129315350 conn(0x7f48b4078240 msgr2=0x7f48b4108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:49.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:49.844+0000 7f48b3e00640 1 -- 172.21.15.159:0/4129315350 shutdown_connections 2024-09-19T07:23:49.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:49.844+0000 7f48b3e00640 1 -- 172.21.15.159:0/4129315350 wait complete. 2024-09-19T07:23:50.604 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T07:23:50.966 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:23:51.104 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:51.104 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:50 smithi159 ceph-mon[30472]: Added host smithi189 2024-09-19T07:23:51.104 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:23:51.104 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:51.105 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:51.105 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:50 smithi159 ceph-mon[30472]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:23:51.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.538+0000 7fa867a00640 1 -- 172.21.15.159:0/2014039410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa860076720 msgr2=0x7fa860076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:51.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.538+0000 7fa867a00640 1 --2- 172.21.15.159:0/2014039410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa860076720 0x7fa860076b00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa854009880 tx=0x7fa85402a5d0 comp rx=0 tx=0).stop 2024-09-19T07:23:51.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.538+0000 7fa867a00640 1 -- 172.21.15.159:0/2014039410 shutdown_connections 2024-09-19T07:23:51.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.538+0000 7fa867a00640 1 --2- 172.21.15.159:0/2014039410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa860076720 0x7fa860076b00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:51.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.538+0000 7fa867a00640 1 -- 172.21.15.159:0/2014039410 >> 172.21.15.159:0/2014039410 conn(0x7fa8600ff4e0 msgr2=0x7fa860101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:51.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.538+0000 7fa867a00640 1 -- 172.21.15.159:0/2014039410 shutdown_connections 2024-09-19T07:23:51.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.538+0000 7fa867a00640 1 -- 172.21.15.159:0/2014039410 wait complete. 2024-09-19T07:23:51.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.539+0000 7fa867a00640 1 Processor -- start 2024-09-19T07:23:51.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.539+0000 7fa867a00640 1 -- start start 2024-09-19T07:23:51.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.539+0000 7fa867a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa860076720 0x7fa8601a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:51.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.539+0000 7fa867a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa8601122d0 con 0x7fa860076720 2024-09-19T07:23:51.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.540+0000 7fa865000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa860076720 0x7fa8601a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:51.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.540+0000 7fa865000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa860076720 0x7fa8601a33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41084/0 (socket says 172.21.15.159:41084) 2024-09-19T07:23:51.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.540+0000 7fa865000640 1 -- 172.21.15.159:0/1707966733 learned_addr learned my addr 172.21.15.159:0/1707966733 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:51.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.540+0000 7fa865000640 1 -- 172.21.15.159:0/1707966733 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8601a3910 con 0x7fa860076720 2024-09-19T07:23:51.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.540+0000 7fa865000640 1 --2- 172.21.15.159:0/1707966733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa860076720 0x7fa8601a33d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa854009230 tx=0x7fa854003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:51.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.541+0000 7fa85d600640 1 -- 172.21.15.159:0/1707966733 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa85403f070 con 0x7fa860076720 2024-09-19T07:23:51.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.541+0000 7fa867a00640 1 -- 172.21.15.159:0/1707966733 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8601a3ba0 con 0x7fa860076720 2024-09-19T07:23:51.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.541+0000 7fa85d600640 1 -- 172.21.15.159:0/1707966733 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa85403bc80 con 0x7fa860076720 2024-09-19T07:23:51.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.541+0000 7fa867a00640 1 -- 172.21.15.159:0/1707966733 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa86019d590 con 0x7fa860076720 2024-09-19T07:23:51.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.541+0000 7fa85d600640 1 -- 172.21.15.159:0/1707966733 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa854049060 con 0x7fa860076720 2024-09-19T07:23:51.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.542+0000 7fa85d600640 1 -- 172.21.15.159:0/1707966733 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fa854042070 con 0x7fa860076720 2024-09-19T07:23:51.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.542+0000 7fa867a00640 1 -- 172.21.15.159:0/1707966733 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa860077e60 con 0x7fa860076720 2024-09-19T07:23:51.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.542+0000 7fa85d600640 1 --2- 172.21.15.159:0/1707966733 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa8480392a0 0x7fa84803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:51.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.542+0000 7fa85d600640 1 -- 172.21.15.159:0/1707966733 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa85406d320 con 0x7fa860076720 2024-09-19T07:23:51.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.543+0000 7fa85fe00640 1 --2- 172.21.15.159:0/1707966733 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa8480392a0 0x7fa84803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:51.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.543+0000 7fa85fe00640 1 --2- 172.21.15.159:0/1707966733 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa8480392a0 0x7fa84803b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0098c0 tx=0x7fa84c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:51.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.552+0000 7fa85d600640 1 -- 172.21.15.159:0/1707966733 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa854031d60 con 0x7fa860076720 2024-09-19T07:23:51.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.734+0000 7fa867a00640 1 -- 172.21.15.159:0/1707966733 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa86019eac0 con 0x7fa8480392a0 2024-09-19T07:23:51.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.735+0000 7fa85d600640 1 -- 172.21.15.159:0/1707966733 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fa86019eac0 con 0x7fa8480392a0 2024-09-19T07:23:51.735 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:23:51.736 INFO:teuthology.orchestra.run.smithi159.stdout:[{"addr": "172.21.15.159", "hostname": "smithi159", "labels": [], "status": ""}, {"addr": "172.21.15.189", "hostname": "smithi189", "labels": [], "status": ""}] 2024-09-19T07:23:51.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.738+0000 7fa867a00640 1 -- 172.21.15.159:0/1707966733 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa8480392a0 msgr2=0x7fa84803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:51.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.738+0000 7fa867a00640 1 --2- 172.21.15.159:0/1707966733 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa8480392a0 0x7fa84803b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0098c0 tx=0x7fa84c006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:51.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.738+0000 7fa867a00640 1 -- 172.21.15.159:0/1707966733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa860076720 msgr2=0x7fa8601a33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.738+0000 7fa867a00640 1 --2- 172.21.15.159:0/1707966733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa860076720 0x7fa8601a33d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa854009230 tx=0x7fa854003d40 comp rx=0 tx=0).stop 2024-09-19T07:23:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.738+0000 7fa867a00640 1 -- 172.21.15.159:0/1707966733 shutdown_connections 2024-09-19T07:23:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.738+0000 7fa867a00640 1 --2- 172.21.15.159:0/1707966733 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa8480392a0 0x7fa84803b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.738+0000 7fa867a00640 1 --2- 172.21.15.159:0/1707966733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa860076720 0x7fa8601a33d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.738+0000 7fa867a00640 1 -- 172.21.15.159:0/1707966733 >> 172.21.15.159:0/1707966733 conn(0x7fa8600ff4e0 msgr2=0x7fa8600ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.738+0000 7fa867a00640 1 -- 172.21.15.159:0/1707966733 shutdown_connections 2024-09-19T07:23:51.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:51.738+0000 7fa867a00640 1 -- 172.21.15.159:0/1707966733 wait complete. 2024-09-19T07:23:52.486 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:52 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:52.486 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:52 smithi159 ceph-mon[30472]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:23:52.523 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi203 2024-09-19T07:23:52.523 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-19T07:23:52.523 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.conf 2024-09-19T07:23:52.543 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-19T07:23:52.543 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:23:52.601 INFO:tasks.cephadm:Adding host smithi203 to orchestrator... 2024-09-19T07:23:52.601 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch host add smithi203 2024-09-19T07:23:52.967 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:23:53.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:53 smithi159 ceph-mon[30472]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:23:53.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.520+0000 7fd73fc00640 1 -- 172.21.15.159:0/1824090376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7381097c0 msgr2=0x7fd738109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:53.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.520+0000 7fd73fc00640 1 --2- 172.21.15.159:0/1824090376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7381097c0 0x7fd738109ba0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009880 tx=0x7fd72c02a5d0 comp rx=0 tx=0).stop 2024-09-19T07:23:53.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.520+0000 7fd73fc00640 1 -- 172.21.15.159:0/1824090376 shutdown_connections 2024-09-19T07:23:53.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.520+0000 7fd73fc00640 1 --2- 172.21.15.159:0/1824090376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7381097c0 0x7fd738109ba0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:53.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.520+0000 7fd73fc00640 1 -- 172.21.15.159:0/1824090376 >> 172.21.15.159:0/1824090376 conn(0x7fd7380ff4e0 msgr2=0x7fd738101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:53.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.520+0000 7fd73fc00640 1 -- 172.21.15.159:0/1824090376 shutdown_connections 2024-09-19T07:23:53.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.520+0000 7fd73fc00640 1 -- 172.21.15.159:0/1824090376 wait complete. 2024-09-19T07:23:53.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.521+0000 7fd73fc00640 1 Processor -- start 2024-09-19T07:23:53.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.522+0000 7fd73fc00640 1 -- start start 2024-09-19T07:23:53.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.522+0000 7fd73fc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7381097c0 0x7fd7381992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:53.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.522+0000 7fd73fc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd73810de80 con 0x7fd7381097c0 2024-09-19T07:23:53.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.522+0000 7fd73d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7381097c0 0x7fd7381992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:53.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.522+0000 7fd73d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7381097c0 0x7fd7381992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33432/0 (socket says 172.21.15.159:33432) 2024-09-19T07:23:53.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.522+0000 7fd73d200640 1 -- 172.21.15.159:0/2155049960 learned_addr learned my addr 172.21.15.159:0/2155049960 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:53.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.523+0000 7fd73d200640 1 -- 172.21.15.159:0/2155049960 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd738199830 con 0x7fd7381097c0 2024-09-19T07:23:53.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.523+0000 7fd73d200640 1 --2- 172.21.15.159:0/2155049960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7381097c0 0x7fd7381992f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009230 tx=0x7fd72c003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:53.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.523+0000 7fd735600640 1 -- 172.21.15.159:0/2155049960 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd72c03f070 con 0x7fd7381097c0 2024-09-19T07:23:53.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.523+0000 7fd735600640 1 -- 172.21.15.159:0/2155049960 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd72c03bd40 con 0x7fd7381097c0 2024-09-19T07:23:53.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.523+0000 7fd735600640 1 -- 172.21.15.159:0/2155049960 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd72c035030 con 0x7fd7381097c0 2024-09-19T07:23:53.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.523+0000 7fd73fc00640 1 -- 172.21.15.159:0/2155049960 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd738199b20 con 0x7fd7381097c0 2024-09-19T07:23:53.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.523+0000 7fd73fc00640 1 -- 172.21.15.159:0/2155049960 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73819dbc0 con 0x7fd7381097c0 2024-09-19T07:23:53.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.525+0000 7fd735600640 1 -- 172.21.15.159:0/2155049960 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fd72c042070 con 0x7fd7381097c0 2024-09-19T07:23:53.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.525+0000 7fd73fc00640 1 -- 172.21.15.159:0/2155049960 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd738104f00 con 0x7fd7381097c0 2024-09-19T07:23:53.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.525+0000 7fd735600640 1 --2- 172.21.15.159:0/2155049960 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd720039300 0x7fd72003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:53.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.525+0000 7fd735600640 1 -- 172.21.15.159:0/2155049960 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd72c06cbb0 con 0x7fd7381097c0 2024-09-19T07:23:53.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.525+0000 7fd737e00640 1 --2- 172.21.15.159:0/2155049960 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd720039300 0x7fd72003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:53.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.526+0000 7fd737e00640 1 --2- 172.21.15.159:0/2155049960 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd720039300 0x7fd72003b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd7240098c0 tx=0x7fd724006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:53.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.530+0000 7fd735600640 1 -- 172.21.15.159:0/2155049960 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd72c044e10 con 0x7fd7381097c0 2024-09-19T07:23:53.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:53.711+0000 7fd73fc00640 1 -- 172.21.15.159:0/2155049960 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}) -- 0x7fd73819a650 con 0x7fd720039300 2024-09-19T07:23:54.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:54 smithi159 ceph-mon[30472]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:23:54.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:54 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:54.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:54 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:54.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:54 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:54.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:54 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:54.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:54 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:23:54.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:54 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:23:54.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:54 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:23:56.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:55 smithi159 ceph-mon[30472]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-19T07:23:56.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:55 smithi159 ceph-mon[30472]: Updating smithi189:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:23:56.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:55 smithi159 ceph-mon[30472]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:23:56.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:55 smithi159 ceph-mon[30472]: Updating smithi189:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:23:56.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:55 smithi159 ceph-mon[30472]: Deploying cephadm binary to smithi203 2024-09-19T07:23:56.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:55 smithi159 ceph-mon[30472]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:23:56.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:55 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:56.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:55 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:56.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:55 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:56.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:56.855+0000 7fd735600640 1 -- 172.21.15.159:0/2155049960 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd73819a650 con 0x7fd720039300 2024-09-19T07:23:56.856 INFO:teuthology.orchestra.run.smithi159.stdout:Added host 'smithi203' with addr '172.21.15.203' 2024-09-19T07:23:56.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:56.858+0000 7fd73fc00640 1 -- 172.21.15.159:0/2155049960 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd720039300 msgr2=0x7fd72003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:56.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:56.858+0000 7fd73fc00640 1 --2- 172.21.15.159:0/2155049960 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd720039300 0x7fd72003b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd7240098c0 tx=0x7fd724006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:56.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:56.858+0000 7fd73fc00640 1 -- 172.21.15.159:0/2155049960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7381097c0 msgr2=0x7fd7381992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:56.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:56.858+0000 7fd73fc00640 1 --2- 172.21.15.159:0/2155049960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7381097c0 0x7fd7381992f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009230 tx=0x7fd72c003d80 comp rx=0 tx=0).stop 2024-09-19T07:23:56.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:56.858+0000 7fd73fc00640 1 -- 172.21.15.159:0/2155049960 shutdown_connections 2024-09-19T07:23:56.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:56.858+0000 7fd73fc00640 1 --2- 172.21.15.159:0/2155049960 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd720039300 0x7fd72003b7c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:56.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:56.858+0000 7fd73fc00640 1 --2- 172.21.15.159:0/2155049960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7381097c0 0x7fd7381992f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:56.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:56.858+0000 7fd73fc00640 1 -- 172.21.15.159:0/2155049960 >> 172.21.15.159:0/2155049960 conn(0x7fd7380ff4e0 msgr2=0x7fd738100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:56.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:56.858+0000 7fd73fc00640 1 -- 172.21.15.159:0/2155049960 shutdown_connections 2024-09-19T07:23:56.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:56.858+0000 7fd73fc00640 1 -- 172.21.15.159:0/2155049960 wait complete. 2024-09-19T07:23:57.697 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T07:23:58.059 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:23:58.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:57 smithi159 ceph-mon[30472]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:23:58.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:58.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:23:58.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:58.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:58.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.604+0000 7f7e1b200640 1 -- 172.21.15.159:0/1465120606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e1c109650 msgr2=0x7f7e1c109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:58.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.604+0000 7f7e1b200640 1 --2- 172.21.15.159:0/1465120606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e1c109650 0x7f7e1c109a30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009c20 tx=0x7f7e0c02aa90 comp rx=0 tx=0).stop 2024-09-19T07:23:58.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.604+0000 7f7e1b200640 1 -- 172.21.15.159:0/1465120606 shutdown_connections 2024-09-19T07:23:58.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.604+0000 7f7e1b200640 1 --2- 172.21.15.159:0/1465120606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e1c109650 0x7f7e1c109a30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:58.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.604+0000 7f7e1b200640 1 -- 172.21.15.159:0/1465120606 >> 172.21.15.159:0/1465120606 conn(0x7f7e1c0ff260 msgr2=0x7f7e1c101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:58.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.604+0000 7f7e1b200640 1 -- 172.21.15.159:0/1465120606 shutdown_connections 2024-09-19T07:23:58.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.604+0000 7f7e1b200640 1 -- 172.21.15.159:0/1465120606 wait complete. 2024-09-19T07:23:58.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.605+0000 7f7e1b200640 1 Processor -- start 2024-09-19T07:23:58.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.605+0000 7f7e1b200640 1 -- start start 2024-09-19T07:23:58.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.606+0000 7f7e1b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e1c109650 0x7f7e1c199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:58.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.606+0000 7f7e1b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1c10dc60 con 0x7f7e1c109650 2024-09-19T07:23:58.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.606+0000 7f7e19e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e1c109650 0x7f7e1c199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:58.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.606+0000 7f7e19e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e1c109650 0x7f7e1c199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33452/0 (socket says 172.21.15.159:33452) 2024-09-19T07:23:58.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.606+0000 7f7e19e00640 1 -- 172.21.15.159:0/4284571566 learned_addr learned my addr 172.21.15.159:0/4284571566 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:23:58.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.606+0000 7f7e19e00640 1 -- 172.21.15.159:0/4284571566 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1c199690 con 0x7f7e1c109650 2024-09-19T07:23:58.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.606+0000 7f7e19e00640 1 --2- 172.21.15.159:0/4284571566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e1c109650 0x7f7e1c199150 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c005e30 tx=0x7f7e0c02fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:58.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.607+0000 7f7e13400640 1 -- 172.21.15.159:0/4284571566 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e0c047070 con 0x7f7e1c109650 2024-09-19T07:23:58.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.607+0000 7f7e13400640 1 -- 172.21.15.159:0/4284571566 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e0c003df0 con 0x7f7e1c109650 2024-09-19T07:23:58.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.607+0000 7f7e1b200640 1 -- 172.21.15.159:0/4284571566 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1c199980 con 0x7f7e1c109650 2024-09-19T07:23:58.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.607+0000 7f7e13400640 1 -- 172.21.15.159:0/4284571566 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e0c033050 con 0x7f7e1c109650 2024-09-19T07:23:58.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.607+0000 7f7e1b200640 1 -- 172.21.15.159:0/4284571566 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e1c19da20 con 0x7f7e1c109650 2024-09-19T07:23:58.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.608+0000 7f7e1b200640 1 -- 172.21.15.159:0/4284571566 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e1c104d00 con 0x7f7e1c109650 2024-09-19T07:23:58.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.608+0000 7f7e13400640 1 -- 172.21.15.159:0/4284571566 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f7e0c04a070 con 0x7f7e1c109650 2024-09-19T07:23:58.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.608+0000 7f7e13400640 1 --2- 172.21.15.159:0/4284571566 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f7e00039300 0x7f7e0003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:23:58.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.609+0000 7f7e13400640 1 -- 172.21.15.159:0/4284571566 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7e0c03e030 con 0x7f7e1c109650 2024-09-19T07:23:58.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.609+0000 7f7e11400640 1 --2- 172.21.15.159:0/4284571566 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f7e00039300 0x7f7e0003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:23:58.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.609+0000 7f7e11400640 1 --2- 172.21.15.159:0/4284571566 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f7e00039300 0x7f7e0003b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7e040098c0 tx=0x7f7e04006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:23:58.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.613+0000 7f7e13400640 1 -- 172.21.15.159:0/4284571566 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7e0c04a360 con 0x7f7e1c109650 2024-09-19T07:23:58.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.798+0000 7f7e1b200640 1 -- 172.21.15.159:0/4284571566 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e1c19a4b0 con 0x7f7e00039300 2024-09-19T07:23:58.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.799+0000 7f7e13400640 1 -- 172.21.15.159:0/4284571566 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f7e1c19a4b0 con 0x7f7e00039300 2024-09-19T07:23:58.800 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:23:58.800 INFO:teuthology.orchestra.run.smithi159.stdout:[{"addr": "172.21.15.159", "hostname": "smithi159", "labels": [], "status": ""}, {"addr": "172.21.15.189", "hostname": "smithi189", "labels": [], "status": ""}, {"addr": "172.21.15.203", "hostname": "smithi203", "labels": [], "status": ""}] 2024-09-19T07:23:58.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.801+0000 7f7e1b200640 1 -- 172.21.15.159:0/4284571566 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f7e00039300 msgr2=0x7f7e0003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:58.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.801+0000 7f7e1b200640 1 --2- 172.21.15.159:0/4284571566 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f7e00039300 0x7f7e0003b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7e040098c0 tx=0x7f7e04006db0 comp rx=0 tx=0).stop 2024-09-19T07:23:58.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.801+0000 7f7e1b200640 1 -- 172.21.15.159:0/4284571566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e1c109650 msgr2=0x7f7e1c199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:23:58.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.801+0000 7f7e1b200640 1 --2- 172.21.15.159:0/4284571566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e1c109650 0x7f7e1c199150 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c005e30 tx=0x7f7e0c02fc10 comp rx=0 tx=0).stop 2024-09-19T07:23:58.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.801+0000 7f7e1b200640 1 -- 172.21.15.159:0/4284571566 shutdown_connections 2024-09-19T07:23:58.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.801+0000 7f7e1b200640 1 --2- 172.21.15.159:0/4284571566 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f7e00039300 0x7f7e0003b7c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:58.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.801+0000 7f7e1b200640 1 --2- 172.21.15.159:0/4284571566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e1c109650 0x7f7e1c199150 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:23:58.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.801+0000 7f7e1b200640 1 -- 172.21.15.159:0/4284571566 >> 172.21.15.159:0/4284571566 conn(0x7f7e1c0ff260 msgr2=0x7f7e1c100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:23:58.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.802+0000 7f7e1b200640 1 -- 172.21.15.159:0/4284571566 shutdown_connections 2024-09-19T07:23:58.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:23:58.802+0000 7f7e1b200640 1 -- 172.21.15.159:0/4284571566 wait complete. 2024-09-19T07:23:59.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:58 smithi159 ceph-mon[30472]: Added host smithi203 2024-09-19T07:23:59.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:58 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:23:59.504 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T07:23:59.504 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T07:23:59.862 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:24:00.024 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:59 smithi159 ceph-mon[30472]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:00.024 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:23:59 smithi159 ceph-mon[30472]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:24:00.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.423+0000 7fdb55400640 1 -- 172.21.15.159:0/337470559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb50109650 msgr2=0x7fdb50109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:00.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.423+0000 7fdb55400640 1 --2- 172.21.15.159:0/337470559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb50109650 0x7fdb50109a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009880 tx=0x7fdb4002a5d0 comp rx=0 tx=0).stop 2024-09-19T07:24:00.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.423+0000 7fdb55400640 1 -- 172.21.15.159:0/337470559 shutdown_connections 2024-09-19T07:24:00.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.423+0000 7fdb55400640 1 --2- 172.21.15.159:0/337470559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb50109650 0x7fdb50109a30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:00.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.423+0000 7fdb55400640 1 -- 172.21.15.159:0/337470559 >> 172.21.15.159:0/337470559 conn(0x7fdb500ff260 msgr2=0x7fdb50101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:00.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.423+0000 7fdb55400640 1 -- 172.21.15.159:0/337470559 shutdown_connections 2024-09-19T07:24:00.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.423+0000 7fdb55400640 1 -- 172.21.15.159:0/337470559 wait complete. 2024-09-19T07:24:00.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.424+0000 7fdb55400640 1 Processor -- start 2024-09-19T07:24:00.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.424+0000 7fdb55400640 1 -- start start 2024-09-19T07:24:00.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.424+0000 7fdb55400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb50109650 0x7fdb50199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:00.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.424+0000 7fdb55400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5010dc60 con 0x7fdb50109650 2024-09-19T07:24:00.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.424+0000 7fdb4fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb50109650 0x7fdb50199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:00.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.424+0000 7fdb4fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb50109650 0x7fdb50199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33474/0 (socket says 172.21.15.159:33474) 2024-09-19T07:24:00.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.424+0000 7fdb4fe00640 1 -- 172.21.15.159:0/4098429054 learned_addr learned my addr 172.21.15.159:0/4098429054 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:24:00.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.425+0000 7fdb4fe00640 1 -- 172.21.15.159:0/4098429054 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb50199640 con 0x7fdb50109650 2024-09-19T07:24:00.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.425+0000 7fdb4fe00640 1 --2- 172.21.15.159:0/4098429054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb50109650 0x7fdb50199100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009230 tx=0x7fdb40003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:00.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.425+0000 7fdb4cc00640 1 -- 172.21.15.159:0/4098429054 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb4003f070 con 0x7fdb50109650 2024-09-19T07:24:00.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.425+0000 7fdb4cc00640 1 -- 172.21.15.159:0/4098429054 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb4003bd40 con 0x7fdb50109650 2024-09-19T07:24:00.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.425+0000 7fdb55400640 1 -- 172.21.15.159:0/4098429054 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb50199930 con 0x7fdb50109650 2024-09-19T07:24:00.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.425+0000 7fdb4cc00640 1 -- 172.21.15.159:0/4098429054 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb40035030 con 0x7fdb50109650 2024-09-19T07:24:00.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.425+0000 7fdb55400640 1 -- 172.21.15.159:0/4098429054 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5019d9d0 con 0x7fdb50109650 2024-09-19T07:24:00.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.426+0000 7fdb55400640 1 -- 172.21.15.159:0/4098429054 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb50104d00 con 0x7fdb50109650 2024-09-19T07:24:00.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.427+0000 7fdb4cc00640 1 -- 172.21.15.159:0/4098429054 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fdb40042070 con 0x7fdb50109650 2024-09-19T07:24:00.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.427+0000 7fdb4cc00640 1 --2- 172.21.15.159:0/4098429054 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdb340392a0 0x7fdb3403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:00.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.427+0000 7fdb4cc00640 1 -- 172.21.15.159:0/4098429054 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdb4006cfb0 con 0x7fdb50109650 2024-09-19T07:24:00.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.427+0000 7fdb4f400640 1 --2- 172.21.15.159:0/4098429054 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdb340392a0 0x7fdb3403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:00.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.428+0000 7fdb4f400640 1 --2- 172.21.15.159:0/4098429054 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdb340392a0 0x7fdb3403b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0098c0 tx=0x7fdb3c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:00.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.432+0000 7fdb4cc00640 1 -- 172.21.15.159:0/4098429054 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdb40030d60 con 0x7fdb50109650 2024-09-19T07:24:00.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.614+0000 7fdb55400640 1 -- 172.21.15.159:0/4098429054 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fdb5019a460 con 0x7fdb50109650 2024-09-19T07:24:00.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.771+0000 7fdb4cc00640 1 -- 172.21.15.159:0/4098429054 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fdb400412c0 con 0x7fdb50109650 2024-09-19T07:24:00.772 INFO:teuthology.orchestra.run.smithi159.stderr:adjusted tunables profile to default 2024-09-19T07:24:00.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.773+0000 7fdb55400640 1 -- 172.21.15.159:0/4098429054 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdb340392a0 msgr2=0x7fdb3403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:00.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.773+0000 7fdb55400640 1 --2- 172.21.15.159:0/4098429054 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdb340392a0 0x7fdb3403b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0098c0 tx=0x7fdb3c006db0 comp rx=0 tx=0).stop 2024-09-19T07:24:00.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.773+0000 7fdb55400640 1 -- 172.21.15.159:0/4098429054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb50109650 msgr2=0x7fdb50199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:00.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.773+0000 7fdb55400640 1 --2- 172.21.15.159:0/4098429054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb50109650 0x7fdb50199100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009230 tx=0x7fdb40003d80 comp rx=0 tx=0).stop 2024-09-19T07:24:00.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.773+0000 7fdb55400640 1 -- 172.21.15.159:0/4098429054 shutdown_connections 2024-09-19T07:24:00.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.773+0000 7fdb55400640 1 --2- 172.21.15.159:0/4098429054 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdb340392a0 0x7fdb3403b760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:00.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.773+0000 7fdb55400640 1 --2- 172.21.15.159:0/4098429054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb50109650 0x7fdb50199100 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:00.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.773+0000 7fdb55400640 1 -- 172.21.15.159:0/4098429054 >> 172.21.15.159:0/4098429054 conn(0x7fdb500ff260 msgr2=0x7fdb50100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:00.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.774+0000 7fdb55400640 1 -- 172.21.15.159:0/4098429054 shutdown_connections 2024-09-19T07:24:00.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:00.774+0000 7fdb55400640 1 -- 172.21.15.159:0/4098429054 wait complete. 2024-09-19T07:24:00.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:00 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/4098429054' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T07:24:00.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:00 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:00.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:00 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:00.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:00 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:00.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:00 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:00.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:00 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:24:00.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:00 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:00.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:00 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:01.594 INFO:tasks.cephadm:Adding mon.a on smithi159 2024-09-19T07:24:01.594 INFO:tasks.cephadm:Adding mon.b on smithi189 2024-09-19T07:24:01.594 INFO:tasks.cephadm:Adding mon.c on smithi203 2024-09-19T07:24:01.595 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi159:172.21.15.159=a;smithi189:172.21.15.189=b;smithi203:172.21.15.203=c' 2024-09-19T07:24:01.831 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:24:01.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:01 smithi159 ceph-mon[30472]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:01.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:01 smithi159 ceph-mon[30472]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-19T07:24:01.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:01 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/4098429054' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T07:24:01.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:01 smithi159 ceph-mon[30472]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:24:01.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:01.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:01.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:02.254 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.253+0000 7fd13ffcb640 1 -- 172.21.15.203:0/1922469349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd138100990 msgr2=0x7fd138100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:02.254 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.253+0000 7fd13ffcb640 1 --2- 172.21.15.203:0/1922469349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd138100990 0x7fd138100d70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd128009c20 tx=0x7fd12802aa90 comp rx=0 tx=0).stop 2024-09-19T07:24:02.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.253+0000 7fd13ffcb640 1 -- 172.21.15.203:0/1922469349 shutdown_connections 2024-09-19T07:24:02.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.253+0000 7fd13ffcb640 1 --2- 172.21.15.203:0/1922469349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd138100990 0x7fd138100d70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:02.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.253+0000 7fd13ffcb640 1 -- 172.21.15.203:0/1922469349 >> 172.21.15.203:0/1922469349 conn(0x7fd138076250 msgr2=0x7fd138076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:02.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.253+0000 7fd13ffcb640 1 -- 172.21.15.203:0/1922469349 shutdown_connections 2024-09-19T07:24:02.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.253+0000 7fd13ffcb640 1 -- 172.21.15.203:0/1922469349 wait complete. 2024-09-19T07:24:02.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.254+0000 7fd13ffcb640 1 Processor -- start 2024-09-19T07:24:02.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.254+0000 7fd13ffcb640 1 -- start start 2024-09-19T07:24:02.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.254+0000 7fd13ffcb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd138100990 0x7fd1381980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:02.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.254+0000 7fd13ffcb640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd13810ab90 con 0x7fd138100990 2024-09-19T07:24:02.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.254+0000 7fd13dd67640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd138100990 0x7fd1381980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:02.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.254+0000 7fd13dd67640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd138100990 0x7fd1381980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.203:52054/0 (socket says 172.21.15.203:52054) 2024-09-19T07:24:02.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.254+0000 7fd13dd67640 1 -- 172.21.15.203:0/2899124702 learned_addr learned my addr 172.21.15.203:0/2899124702 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-19T07:24:02.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.255+0000 7fd13dd67640 1 -- 172.21.15.203:0/2899124702 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1381985e0 con 0x7fd138100990 2024-09-19T07:24:02.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.255+0000 7fd13dd67640 1 --2- 172.21.15.203:0/2899124702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd138100990 0x7fd1381980a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd128005e30 tx=0x7fd12802fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:02.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.255+0000 7fd136ffd640 1 -- 172.21.15.203:0/2899124702 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd128047070 con 0x7fd138100990 2024-09-19T07:24:02.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.255+0000 7fd13ffcb640 1 -- 172.21.15.203:0/2899124702 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1381988d0 con 0x7fd138100990 2024-09-19T07:24:02.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.255+0000 7fd136ffd640 1 -- 172.21.15.203:0/2899124702 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd128003df0 con 0x7fd138100990 2024-09-19T07:24:02.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.255+0000 7fd136ffd640 1 -- 172.21.15.203:0/2899124702 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd128033050 con 0x7fd138100990 2024-09-19T07:24:02.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.255+0000 7fd13ffcb640 1 -- 172.21.15.203:0/2899124702 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd138198d90 con 0x7fd138100990 2024-09-19T07:24:02.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.256+0000 7fd136ffd640 1 -- 172.21.15.203:0/2899124702 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fd12804a070 con 0x7fd138100990 2024-09-19T07:24:02.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.256+0000 7fd13ffcb640 1 -- 172.21.15.203:0/2899124702 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd138101c40 con 0x7fd138100990 2024-09-19T07:24:02.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.256+0000 7fd136ffd640 1 --2- 172.21.15.203:0/2899124702 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd1200392a0 0x7fd12003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:02.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.256+0000 7fd136ffd640 1 -- 172.21.15.203:0/2899124702 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd12803e030 con 0x7fd138100990 2024-09-19T07:24:02.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.257+0000 7fd13d566640 1 --2- 172.21.15.203:0/2899124702 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd1200392a0 0x7fd12003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:02.258 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.257+0000 7fd13d566640 1 --2- 172.21.15.203:0/2899124702 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd1200392a0 0x7fd12003b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0098c0 tx=0x7fd12c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:02.262 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.262+0000 7fd136ffd640 1 -- 172.21.15.203:0/2899124702 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd12804a360 con 0x7fd138100990 2024-09-19T07:24:02.440 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.440+0000 7fd13ffcb640 1 -- 172.21.15.203:0/2899124702 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi159:172.21.15.159=a;smithi189:172.21.15.189=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}) -- 0x7fd138069cb0 con 0x7fd1200392a0 2024-09-19T07:24:02.446 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.446+0000 7fd136ffd640 1 -- 172.21.15.203:0/2899124702 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd138069cb0 con 0x7fd1200392a0 2024-09-19T07:24:02.446 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mon update... 2024-09-19T07:24:02.449 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.448+0000 7fd13ffcb640 1 -- 172.21.15.203:0/2899124702 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd1200392a0 msgr2=0x7fd12003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:02.449 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.448+0000 7fd13ffcb640 1 --2- 172.21.15.203:0/2899124702 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd1200392a0 0x7fd12003b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0098c0 tx=0x7fd12c006db0 comp rx=0 tx=0).stop 2024-09-19T07:24:02.449 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.448+0000 7fd13ffcb640 1 -- 172.21.15.203:0/2899124702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd138100990 msgr2=0x7fd1381980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:02.449 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.448+0000 7fd13ffcb640 1 --2- 172.21.15.203:0/2899124702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd138100990 0x7fd1381980a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd128005e30 tx=0x7fd12802fc10 comp rx=0 tx=0).stop 2024-09-19T07:24:02.449 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.448+0000 7fd13ffcb640 1 -- 172.21.15.203:0/2899124702 shutdown_connections 2024-09-19T07:24:02.450 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.448+0000 7fd13ffcb640 1 --2- 172.21.15.203:0/2899124702 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd1200392a0 0x7fd12003b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:02.450 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.448+0000 7fd13ffcb640 1 --2- 172.21.15.203:0/2899124702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd138100990 0x7fd1381980a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:02.450 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.448+0000 7fd13ffcb640 1 -- 172.21.15.203:0/2899124702 >> 172.21.15.203:0/2899124702 conn(0x7fd138076250 msgr2=0x7fd1381020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:02.450 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.448+0000 7fd13ffcb640 1 -- 172.21.15.203:0/2899124702 shutdown_connections 2024-09-19T07:24:02.450 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:02.448+0000 7fd13ffcb640 1 -- 172.21.15.203:0/2899124702 wait complete. 2024-09-19T07:24:03.027 DEBUG:teuthology.orchestra.run.smithi189:mon.b> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mon.b.service 2024-09-19T07:24:03.030 DEBUG:teuthology.orchestra.run.smithi203:mon.c> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mon.c.service 2024-09-19T07:24:03.032 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T07:24:03.032 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T07:24:03.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:02 smithi159 ceph-mon[30472]: Updating smithi203:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:24:03.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:02 smithi159 ceph-mon[30472]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:24:03.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:02 smithi159 ceph-mon[30472]: Updating smithi203:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:24:03.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:02 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:03.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:02 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:03.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:02 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:03.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:02 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:03.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:02 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:03.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:02 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:24:03.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:02 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:03.697 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:24:04.094 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.093+0000 7f2f3ce64640 1 -- 172.21.15.203:0/368532659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f38107660 msgr2=0x7f2f38107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:04.095 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.093+0000 7f2f3ce64640 1 --2- 172.21.15.203:0/368532659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f38107660 0x7f2f38107a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009c20 tx=0x7f2f2802aa90 comp rx=0 tx=0).stop 2024-09-19T07:24:04.095 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.093+0000 7f2f3ce64640 1 -- 172.21.15.203:0/368532659 shutdown_connections 2024-09-19T07:24:04.095 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.093+0000 7f2f3ce64640 1 --2- 172.21.15.203:0/368532659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f38107660 0x7f2f38107a40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:04.095 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.093+0000 7f2f3ce64640 1 -- 172.21.15.203:0/368532659 >> 172.21.15.203:0/368532659 conn(0x7f2f380fd2b0 msgr2=0x7f2f380ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:04.095 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.094+0000 7f2f3ce64640 1 -- 172.21.15.203:0/368532659 shutdown_connections 2024-09-19T07:24:04.095 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.094+0000 7f2f3ce64640 1 -- 172.21.15.203:0/368532659 wait complete. 2024-09-19T07:24:04.095 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.094+0000 7f2f3ce64640 1 Processor -- start 2024-09-19T07:24:04.095 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.094+0000 7f2f3ce64640 1 -- start start 2024-09-19T07:24:04.095 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.095+0000 7f2f3ce64640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f38107660 0x7f2f381970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:04.095 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.095+0000 7f2f3ce64640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3810bc70 con 0x7f2f38107660 2024-09-19T07:24:04.096 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.095+0000 7f2f3659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f38107660 0x7f2f381970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:04.096 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.095+0000 7f2f3659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f38107660 0x7f2f381970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.203:52078/0 (socket says 172.21.15.203:52078) 2024-09-19T07:24:04.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.095+0000 7f2f3659c640 1 -- 172.21.15.203:0/1642955359 learned_addr learned my addr 172.21.15.203:0/1642955359 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-19T07:24:04.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.095+0000 7f2f3659c640 1 -- 172.21.15.203:0/1642955359 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f381975e0 con 0x7f2f38107660 2024-09-19T07:24:04.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.096+0000 7f2f3659c640 1 --2- 172.21.15.203:0/1642955359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f38107660 0x7f2f381970a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009d40 tx=0x7f2f2802fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:04.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.096+0000 7f2f277fe640 1 -- 172.21.15.203:0/1642955359 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f28047070 con 0x7f2f38107660 2024-09-19T07:24:04.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.096+0000 7f2f3ce64640 1 -- 172.21.15.203:0/1642955359 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f38197870 con 0x7f2f38107660 2024-09-19T07:24:04.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.096+0000 7f2f277fe640 1 -- 172.21.15.203:0/1642955359 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f28003e40 con 0x7f2f38107660 2024-09-19T07:24:04.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.096+0000 7f2f277fe640 1 -- 172.21.15.203:0/1642955359 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f28033050 con 0x7f2f38107660 2024-09-19T07:24:04.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.096+0000 7f2f3ce64640 1 -- 172.21.15.203:0/1642955359 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f3819b970 con 0x7f2f38107660 2024-09-19T07:24:04.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.097+0000 7f2f277fe640 1 -- 172.21.15.203:0/1642955359 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f2f2804a070 con 0x7f2f38107660 2024-09-19T07:24:04.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.097+0000 7f2f3ce64640 1 -- 172.21.15.203:0/1642955359 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f38102d10 con 0x7f2f38107660 2024-09-19T07:24:04.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.097+0000 7f2f277fe640 1 --2- 172.21.15.203:0/1642955359 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2f200392a0 0x7f2f2003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:04.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.097+0000 7f2f277fe640 1 -- 172.21.15.203:0/1642955359 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2f2803e030 con 0x7f2f38107660 2024-09-19T07:24:04.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.098+0000 7f2f35d9b640 1 --2- 172.21.15.203:0/1642955359 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2f200392a0 0x7f2f2003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:04.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.099+0000 7f2f35d9b640 1 --2- 172.21.15.203:0/1642955359 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2f200392a0 0x7f2f2003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0098c0 tx=0x7f2f2c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:04.103 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.103+0000 7f2f277fe640 1 -- 172.21.15.203:0/1642955359 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2f28004180 con 0x7f2f38107660 2024-09-19T07:24:04.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:03 smithi159 ceph-mon[30472]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi159:172.21.15.159=a;smithi189:172.21.15.189=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:24:04.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:03 smithi159 ceph-mon[30472]: Saving service mon spec with placement smithi159:172.21.15.159=a;smithi189:172.21.15.189=b;smithi203:172.21.15.203=c;count:3 2024-09-19T07:24:04.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:03 smithi159 ceph-mon[30472]: Deploying daemon mon.c on smithi203 2024-09-19T07:24:04.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:03 smithi159 ceph-mon[30472]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:04.336 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.335+0000 7f2f3ce64640 1 -- 172.21.15.203:0/1642955359 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2f38198400 con 0x7f2f38107660 2024-09-19T07:24:04.336 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.336+0000 7f2f277fe640 1 -- 172.21.15.203:0/1642955359 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f2f28051300 con 0x7f2f38107660 2024-09-19T07:24:04.336 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:24:04.337 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"c9d18048-7657-11ef-bceb-c7b262605968","modified":"2024-09-19T07:22:28.312056Z","created":"2024-09-19T07:22:28.312056Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T07:24:04.341 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-09-19T07:24:04.341 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.338+0000 7f2f3ce64640 1 -- 172.21.15.203:0/1642955359 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2f200392a0 msgr2=0x7f2f2003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:04.341 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.338+0000 7f2f3ce64640 1 --2- 172.21.15.203:0/1642955359 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2f200392a0 0x7f2f2003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0098c0 tx=0x7f2f2c006db0 comp rx=0 tx=0).stop 2024-09-19T07:24:04.341 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.338+0000 7f2f3ce64640 1 -- 172.21.15.203:0/1642955359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f38107660 msgr2=0x7f2f381970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:04.341 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.338+0000 7f2f3ce64640 1 --2- 172.21.15.203:0/1642955359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f38107660 0x7f2f381970a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009d40 tx=0x7f2f2802fc10 comp rx=0 tx=0).stop 2024-09-19T07:24:04.341 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.338+0000 7f2f3ce64640 1 -- 172.21.15.203:0/1642955359 shutdown_connections 2024-09-19T07:24:04.341 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.338+0000 7f2f3ce64640 1 --2- 172.21.15.203:0/1642955359 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2f200392a0 0x7f2f2003b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:04.341 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.338+0000 7f2f3ce64640 1 --2- 172.21.15.203:0/1642955359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f38107660 0x7f2f381970a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:04.341 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.338+0000 7f2f3ce64640 1 -- 172.21.15.203:0/1642955359 >> 172.21.15.203:0/1642955359 conn(0x7f2f380fd2b0 msgr2=0x7f2f380feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:04.341 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.338+0000 7f2f3ce64640 1 -- 172.21.15.203:0/1642955359 shutdown_connections 2024-09-19T07:24:04.341 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:04.338+0000 7f2f3ce64640 1 -- 172.21.15.203:0/1642955359 wait complete. 2024-09-19T07:24:05.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:04 smithi159 ceph-mon[30472]: from='client.? 172.21.15.203:0/1642955359' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:24:06.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:05 smithi159 ceph-mon[30472]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:07.182 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T07:24:07.182 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T07:24:07.435 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:24:12.280 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.277+0000 7f9cf9994640 1 -- 172.21.15.203:0/1278969529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ce0003530 msgr2=0x7f9ce00059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:12.280 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.277+0000 7f9cf9994640 1 --2- 172.21.15.203:0/1278969529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ce0003530 0x7f9ce00059c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4102af0 tx=0x7f9cec02bfe0 comp rx=0 tx=0).stop 2024-09-19T07:24:12.280 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.277+0000 7f9cf9994640 1 -- 172.21.15.203:0/1278969529 shutdown_connections 2024-09-19T07:24:12.281 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.277+0000 7f9cf9994640 1 --2- 172.21.15.203:0/1278969529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ce0003530 0x7f9ce00059c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:12.281 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.277+0000 7f9cf9994640 1 -- 172.21.15.203:0/1278969529 >> 172.21.15.203:0/1278969529 conn(0x7f9cf40fd320 msgr2=0x7f9cf40ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:12.281 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.277+0000 7f9cf9994640 1 -- 172.21.15.203:0/1278969529 shutdown_connections 2024-09-19T07:24:12.281 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.277+0000 7f9cf9994640 1 -- 172.21.15.203:0/1278969529 wait complete. 2024-09-19T07:24:12.281 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.278+0000 7f9cf9994640 1 Processor -- start 2024-09-19T07:24:12.281 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.278+0000 7f9cf9994640 1 -- start start 2024-09-19T07:24:12.281 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.279+0000 7f9cf9994640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ce0003530 0x7f9cf4197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:12.281 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.279+0000 7f9cf9994640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cf4197760 0x7f9cf419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:12.281 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.279+0000 7f9cf9994640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf4109780 con 0x7f9cf4197760 2024-09-19T07:24:12.281 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.279+0000 7f9cf9994640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf4109600 con 0x7f9ce0003530 2024-09-19T07:24:12.281 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.279+0000 7f9cf2ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ce0003530 0x7f9cf4197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:12.281 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.279+0000 7f9cf2ffd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ce0003530 0x7f9cf4197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:55342/0 (socket says 172.21.15.203:55342) 2024-09-19T07:24:12.281 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.279+0000 7f9cf2ffd640 1 -- 172.21.15.203:0/1130573377 learned_addr learned my addr 172.21.15.203:0/1130573377 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-19T07:24:12.282 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.279+0000 7f9cf2ffd640 1 -- 172.21.15.203:0/1130573377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ce0003530 msgr2=0x7f9cf4197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T07:24:12.282 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.279+0000 7f9cf2ffd640 1 -- 172.21.15.203:0/1130573377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ce0003530 msgr2=0x7f9cf4197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T07:24:12.282 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.279+0000 7f9cf2ffd640 1 --2- 172.21.15.203:0/1130573377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ce0003530 0x7f9cf4197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T07:24:12.282 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.279+0000 7f9cf2ffd640 1 --2- 172.21.15.203:0/1130573377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ce0003530 0x7f9cf4197220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:24:12.282 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.279+0000 7f9cf27fc640 1 --2- 172.21.15.203:0/1130573377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cf4197760 0x7f9cf419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:12.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.280+0000 7f9cf27fc640 1 -- 172.21.15.203:0/1130573377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ce0003530 msgr2=0x7f9cf4197220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:24:12.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.280+0000 7f9cf27fc640 1 --2- 172.21.15.203:0/1130573377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ce0003530 0x7f9cf4197220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:12.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.280+0000 7f9cf27fc640 1 -- 172.21.15.203:0/1130573377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf4197d80 con 0x7f9cf4197760 2024-09-19T07:24:12.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.280+0000 7f9cf27fc640 1 --2- 172.21.15.203:0/1130573377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cf4197760 0x7f9cf419baf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00b3d0 tx=0x7f9cdc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:12.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.280+0000 7f9cf8992640 1 -- 172.21.15.203:0/1130573377 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9cdc011630 con 0x7f9cf4197760 2024-09-19T07:24:12.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.280+0000 7f9cf8992640 1 -- 172.21.15.203:0/1130573377 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cdc011cb0 con 0x7f9cf4197760 2024-09-19T07:24:12.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.280+0000 7f9cf9994640 1 -- 172.21.15.203:0/1130573377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf419c030 con 0x7f9cf4197760 2024-09-19T07:24:12.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.280+0000 7f9cf8992640 1 -- 172.21.15.203:0/1130573377 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9cdc010df0 con 0x7f9cf4197760 2024-09-19T07:24:12.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.280+0000 7f9cf9994640 1 -- 172.21.15.203:0/1130573377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf419c570 con 0x7f9cf4197760 2024-09-19T07:24:12.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.281+0000 7f9cf8992640 1 -- 172.21.15.203:0/1130573377 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f9cdc018020 con 0x7f9cf4197760 2024-09-19T07:24:12.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.282+0000 7f9cf9994640 1 -- 172.21.15.203:0/1130573377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf4102da0 con 0x7f9cf4197760 2024-09-19T07:24:12.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.282+0000 7f9cf8992640 1 --2- 172.21.15.203:0/1130573377 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9cd4039090 0x7f9cd403b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:12.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.282+0000 7f9cf8992640 1 -- 172.21.15.203:0/1130573377 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9cdc04bec0 con 0x7f9cf4197760 2024-09-19T07:24:12.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.282+0000 7f9cf2ffd640 1 --2- 172.21.15.203:0/1130573377 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9cd4039090 0x7f9cd403b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:12.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.283+0000 7f9cf2ffd640 1 --2- 172.21.15.203:0/1130573377 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9cd4039090 0x7f9cd403b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009b30 tx=0x7f9cec005f60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:12.288 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.287+0000 7f9cf8992640 1 -- 172.21.15.203:0/1130573377 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9cdc01f020 con 0x7f9cf4197760 2024-09-19T07:24:12.519 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.518+0000 7f9cf9994640 1 -- 172.21.15.203:0/1130573377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9cf4190f10 con 0x7f9cf4197760 2024-09-19T07:24:12.520 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.519+0000 7f9cf8992640 1 -- 172.21.15.203:0/1130573377 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f9cdc018310 con 0x7f9cf4197760 2024-09-19T07:24:12.520 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:24:12.520 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":2,"fsid":"c9d18048-7657-11ef-bceb-c7b262605968","modified":"2024-09-19T07:24:07.174125Z","created":"2024-09-19T07:22:28.312056Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T07:24:12.520 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 2 2024-09-19T07:24:12.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.521+0000 7f9cf9994640 1 -- 172.21.15.203:0/1130573377 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9cd4039090 msgr2=0x7f9cd403b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:12.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.521+0000 7f9cf9994640 1 --2- 172.21.15.203:0/1130573377 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9cd4039090 0x7f9cd403b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009b30 tx=0x7f9cec005f60 comp rx=0 tx=0).stop 2024-09-19T07:24:12.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.521+0000 7f9cf9994640 1 -- 172.21.15.203:0/1130573377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cf4197760 msgr2=0x7f9cf419baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:12.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.521+0000 7f9cf9994640 1 --2- 172.21.15.203:0/1130573377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cf4197760 0x7f9cf419baf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00b3d0 tx=0x7f9cdc00b890 comp rx=0 tx=0).stop 2024-09-19T07:24:12.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.521+0000 7f9cf9994640 1 -- 172.21.15.203:0/1130573377 shutdown_connections 2024-09-19T07:24:12.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.521+0000 7f9cf9994640 1 --2- 172.21.15.203:0/1130573377 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9cd4039090 0x7f9cd403b550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:12.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.521+0000 7f9cf9994640 1 --2- 172.21.15.203:0/1130573377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cf4197760 0x7f9cf419baf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:12.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.521+0000 7f9cf9994640 1 --2- 172.21.15.203:0/1130573377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ce0003530 0x7f9cf4197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:12.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.521+0000 7f9cf9994640 1 -- 172.21.15.203:0/1130573377 >> 172.21.15.203:0/1130573377 conn(0x7f9cf40fd320 msgr2=0x7f9cf40ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:12.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.521+0000 7f9cf9994640 1 -- 172.21.15.203:0/1130573377 shutdown_connections 2024-09-19T07:24:12.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:12.521+0000 7f9cf9994640 1 -- 172.21.15.203:0/1130573377 wait complete. 2024-09-19T07:24:12.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:24:12.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:12.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: mon.a calling monitor election 2024-09-19T07:24:12.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:12.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:12.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:12.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: mon.c calling monitor election 2024-09-19T07:24:12.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:12.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:12.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:12.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:12.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T07:24:12.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: monmap epoch 2 2024-09-19T07:24:12.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: fsid c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:24:12.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: last_changed 2024-09-19T07:24:07.174125+0000 2024-09-19T07:24:12.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: created 2024-09-19T07:22:28.312056+0000 2024-09-19T07:24:12.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: min_mon_release 19 (squid) 2024-09-19T07:24:12.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: election_strategy: 1 2024-09-19T07:24:12.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: 0: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.a 2024-09-19T07:24:12.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-09-19T07:24:12.639 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: fsmap 2024-09-19T07:24:12.639 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:24:12.639 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: mgrmap e13: x(active, since 41s) 2024-09-19T07:24:12.639 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: overall HEALTH_OK 2024-09-19T07:24:12.639 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:12.639 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:12.639 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:12.639 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:24:12.639 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:13.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:13 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:13.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:13 smithi159 ceph-mon[30472]: Deploying daemon mon.b on smithi189 2024-09-19T07:24:13.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:13 smithi159 ceph-mon[30472]: from='client.? 172.21.15.203:0/1130573377' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:24:13.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:13 smithi159 ceph-mon[30472]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:13.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:13 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:14.108 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T07:24:14.108 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T07:24:14.366 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:24:14.637 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:24:14 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:24:14.173+0000 7f0886600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-19T07:24:14.769 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.768+0000 7f8dd01cd640 1 -- 172.21.15.203:0/2154149144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8dc8102820 msgr2=0x7f8dc8102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:14.769 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.768+0000 7f8dd01cd640 1 --2- 172.21.15.203:0/2154149144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8dc8102820 0x7f8dc8102c00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4009880 tx=0x7f8dc402a670 comp rx=0 tx=0).stop 2024-09-19T07:24:14.769 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.768+0000 7f8dd01cd640 1 -- 172.21.15.203:0/2154149144 shutdown_connections 2024-09-19T07:24:14.769 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.768+0000 7f8dd01cd640 1 --2- 172.21.15.203:0/2154149144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8dc8102820 0x7f8dc8102c00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:14.769 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.768+0000 7f8dd01cd640 1 -- 172.21.15.203:0/2154149144 >> 172.21.15.203:0/2154149144 conn(0x7f8dc80fc220 msgr2=0x7f8dc80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:14.769 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.768+0000 7f8dd01cd640 1 -- 172.21.15.203:0/2154149144 shutdown_connections 2024-09-19T07:24:14.769 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.768+0000 7f8dd01cd640 1 -- 172.21.15.203:0/2154149144 wait complete. 2024-09-19T07:24:14.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.769+0000 7f8dd01cd640 1 Processor -- start 2024-09-19T07:24:14.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.769+0000 7f8dd01cd640 1 -- start start 2024-09-19T07:24:14.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.769+0000 7f8dd01cd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc8102820 0x7f8dc8100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:14.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.769+0000 7f8dd01cd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8dc81009c0 0x7f8dc8100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:14.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.769+0000 7f8dd01cd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc810bd70 con 0x7f8dc81009c0 2024-09-19T07:24:14.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.769+0000 7f8dd01cd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc810bbf0 con 0x7f8dc8102820 2024-09-19T07:24:14.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.769+0000 7f8dcdf69640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc8102820 0x7f8dc8100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:14.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.769+0000 7f8dcdf69640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc8102820 0x7f8dc8100480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:55350/0 (socket says 172.21.15.203:55350) 2024-09-19T07:24:14.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.769+0000 7f8dcdf69640 1 -- 172.21.15.203:0/2643022009 learned_addr learned my addr 172.21.15.203:0/2643022009 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-19T07:24:14.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.769+0000 7f8dcdf69640 1 -- 172.21.15.203:0/2643022009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc8102820 msgr2=0x7f8dc8100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T07:24:14.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.769+0000 7f8dcdf69640 1 -- 172.21.15.203:0/2643022009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc8102820 msgr2=0x7f8dc8100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T07:24:14.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.769+0000 7f8dcdf69640 1 --2- 172.21.15.203:0/2643022009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc8102820 0x7f8dc8100480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T07:24:14.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.769+0000 7f8dcdf69640 1 --2- 172.21.15.203:0/2643022009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc8102820 0x7f8dc8100480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:24:14.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.769+0000 7f8dcd768640 1 --2- 172.21.15.203:0/2643022009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8dc81009c0 0x7f8dc8100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:14.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.770+0000 7f8dcd768640 1 -- 172.21.15.203:0/2643022009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc8102820 msgr2=0x7f8dc8100480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:24:14.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.770+0000 7f8dcd768640 1 --2- 172.21.15.203:0/2643022009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc8102820 0x7f8dc8100480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:14.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.770+0000 7f8dcd768640 1 -- 172.21.15.203:0/2643022009 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc8102370 con 0x7f8dc81009c0 2024-09-19T07:24:14.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.770+0000 7f8dcd768640 1 --2- 172.21.15.203:0/2643022009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8dc81009c0 0x7f8dc8100e20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8db800b400 tx=0x7f8db800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:14.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.770+0000 7f8dbeffd640 1 -- 172.21.15.203:0/2643022009 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8db8011630 con 0x7f8dc81009c0 2024-09-19T07:24:14.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.770+0000 7f8dd01cd640 1 -- 172.21.15.203:0/2643022009 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc81a36c0 con 0x7f8dc81009c0 2024-09-19T07:24:14.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.770+0000 7f8dbeffd640 1 -- 172.21.15.203:0/2643022009 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8db8011cb0 con 0x7f8dc81009c0 2024-09-19T07:24:14.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.770+0000 7f8dbeffd640 1 -- 172.21.15.203:0/2643022009 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8db8010d80 con 0x7f8dc81009c0 2024-09-19T07:24:14.772 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.770+0000 7f8dd01cd640 1 -- 172.21.15.203:0/2643022009 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc81a3c50 con 0x7f8dc81009c0 2024-09-19T07:24:14.773 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.771+0000 7f8dbeffd640 1 -- 172.21.15.203:0/2643022009 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f8db8018020 con 0x7f8dc81009c0 2024-09-19T07:24:14.773 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.771+0000 7f8dd01cd640 1 -- 172.21.15.203:0/2643022009 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc806ad10 con 0x7f8dc81009c0 2024-09-19T07:24:14.773 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.771+0000 7f8dbeffd640 1 --2- 172.21.15.203:0/2643022009 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8db4039030 0x7f8db403b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:14.773 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.772+0000 7f8dbeffd640 1 -- 172.21.15.203:0/2643022009 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8db804bfa0 con 0x7f8dc81009c0 2024-09-19T07:24:14.774 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.772+0000 7f8dcdf69640 1 --2- 172.21.15.203:0/2643022009 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8db4039030 0x7f8db403b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:14.774 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.772+0000 7f8dcdf69640 1 --2- 172.21.15.203:0/2643022009 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8db4039030 0x7f8db403b4f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40096c0 tx=0x7f8dc4005a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:14.777 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:14.777+0000 7f8dbeffd640 1 -- 172.21.15.203:0/2643022009 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8db801f020 con 0x7f8dc81009c0 2024-09-19T07:24:15.007 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:15.007+0000 7f8dd01cd640 1 -- 172.21.15.203:0/2643022009 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8dc81a3f60 con 0x7f8dc81009c0 2024-09-19T07:24:15.008 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:15.008+0000 7f8dbeffd640 1 -- 172.21.15.203:0/2643022009 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f8db80172c0 con 0x7f8dc81009c0 2024-09-19T07:24:15.008 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:24:15.008 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":2,"fsid":"c9d18048-7657-11ef-bceb-c7b262605968","modified":"2024-09-19T07:24:07.174125Z","created":"2024-09-19T07:22:28.312056Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T07:24:15.009 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 2 2024-09-19T07:24:15.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:15.010+0000 7f8dd01cd640 1 -- 172.21.15.203:0/2643022009 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8db4039030 msgr2=0x7f8db403b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:15.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:15.010+0000 7f8dd01cd640 1 --2- 172.21.15.203:0/2643022009 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8db4039030 0x7f8db403b4f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40096c0 tx=0x7f8dc4005a40 comp rx=0 tx=0).stop 2024-09-19T07:24:15.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:15.010+0000 7f8dd01cd640 1 -- 172.21.15.203:0/2643022009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8dc81009c0 msgr2=0x7f8dc8100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:15.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:15.010+0000 7f8dd01cd640 1 --2- 172.21.15.203:0/2643022009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8dc81009c0 0x7f8dc8100e20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8db800b400 tx=0x7f8db800b8c0 comp rx=0 tx=0).stop 2024-09-19T07:24:15.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:15.010+0000 7f8dd01cd640 1 -- 172.21.15.203:0/2643022009 shutdown_connections 2024-09-19T07:24:15.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:15.010+0000 7f8dd01cd640 1 --2- 172.21.15.203:0/2643022009 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8db4039030 0x7f8db403b4f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:15.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:15.010+0000 7f8dd01cd640 1 --2- 172.21.15.203:0/2643022009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8dc81009c0 0x7f8dc8100e20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:15.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:15.010+0000 7f8dd01cd640 1 --2- 172.21.15.203:0/2643022009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dc8102820 0x7f8dc8100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:15.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:15.010+0000 7f8dd01cd640 1 -- 172.21.15.203:0/2643022009 >> 172.21.15.203:0/2643022009 conn(0x7f8dc80fc220 msgr2=0x7f8dc81055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:15.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:15.010+0000 7f8dd01cd640 1 -- 172.21.15.203:0/2643022009 shutdown_connections 2024-09-19T07:24:15.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:15.010+0000 7f8dd01cd640 1 -- 172.21.15.203:0/2643022009 wait complete. 2024-09-19T07:24:15.616 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:15 smithi189 systemd[1]: Starting Ceph mon.b for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:24:15.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:15 smithi203 ceph-mon[35335]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:15.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:15 smithi203 ceph-mon[35335]: from='client.? 172.21.15.203:0/2643022009' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:24:15.890 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:15 smithi189 podman[34833]: 2024-09-19 07:24:15.777806033 +0000 UTC m=+0.086630067 container create e3b690676dfcb4ef321000fdd584730a6dda23ed638567563faa90e0dc3e0714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:24:15.891 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:15 smithi189 podman[34833]: 2024-09-19 07:24:15.707304974 +0000 UTC m=+0.016129022 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:24:16.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:15 smithi159 ceph-mon[30472]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:16.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:15 smithi159 ceph-mon[30472]: from='client.? 172.21.15.203:0/2643022009' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:24:16.159 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:15 smithi189 podman[34833]: 2024-09-19 07:24:15.995321368 +0000 UTC m=+0.304145401 container init e3b690676dfcb4ef321000fdd584730a6dda23ed638567563faa90e0dc3e0714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-19T07:24:16.160 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:15 smithi189 podman[34833]: 2024-09-19 07:24:15.998641025 +0000 UTC m=+0.307465060 container start e3b690676dfcb4ef321000fdd584730a6dda23ed638567563faa90e0dc3e0714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-b, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T07:24:16.160 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T07:24:16.160 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T07:24:16.160 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: pidfile_write: ignore empty --pid-file 2024-09-19T07:24:16.160 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: load: jerasure load: lrc 2024-09-19T07:24:16.160 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: RocksDB version: 7.9.2 2024-09-19T07:24:16.160 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Git sha 0 2024-09-19T07:24:16.160 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T07:24:16.160 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: DB SUMMARY 2024-09-19T07:24:16.161 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: DB Session ID: 6WJBA84RX07GRJUMH32L 2024-09-19T07:24:16.161 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: CURRENT file: CURRENT 2024-09-19T07:24:16.161 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T07:24:16.161 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-19T07:24:16.161 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-19T07:24:16.161 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-19T07:24:16.161 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.error_if_exists: 0 2024-09-19T07:24:16.161 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.create_if_missing: 0 2024-09-19T07:24:16.161 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.paranoid_checks: 1 2024-09-19T07:24:16.161 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T07:24:16.161 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T07:24:16.162 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T07:24:16.162 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.env: 0x556255099940 2024-09-19T07:24:16.162 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T07:24:16.162 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.info_log: 0x5562566136e0 2024-09-19T07:24:16.162 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T07:24:16.164 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.statistics: (nil) 2024-09-19T07:24:16.164 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.use_fsync: 0 2024-09-19T07:24:16.164 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_log_file_size: 0 2024-09-19T07:24:16.164 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T07:24:16.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T07:24:16.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T07:24:16.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T07:24:16.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.allow_fallocate: 1 2024-09-19T07:24:16.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T07:24:16.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T07:24:16.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.use_direct_reads: 0 2024-09-19T07:24:16.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T07:24:16.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T07:24:16.165 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.db_log_dir: 2024-09-19T07:24:16.166 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.wal_dir: 2024-09-19T07:24:16.166 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T07:24:16.166 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T07:24:16.166 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T07:24:16.166 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T07:24:16.166 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T07:24:16.166 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T07:24:16.166 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T07:24:16.166 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T07:24:16.166 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.write_buffer_manager: 0x556256622dc0 2024-09-19T07:24:16.166 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T07:24:16.166 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T07:24:16.167 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T07:24:16.167 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T07:24:16.167 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T07:24:16.167 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T07:24:16.167 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T07:24:16.167 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T07:24:16.167 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.unordered_write: 0 2024-09-19T07:24:16.167 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T07:24:16.167 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T07:24:16.167 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T07:24:16.167 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T07:24:16.167 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.row_cache: None 2024-09-19T07:24:16.167 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.wal_filter: None 2024-09-19T07:24:16.168 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T07:24:16.168 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T07:24:16.168 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.two_write_queues: 0 2024-09-19T07:24:16.168 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T07:24:16.168 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.wal_compression: 0 2024-09-19T07:24:16.168 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.atomic_flush: 0 2024-09-19T07:24:16.168 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T07:24:16.168 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T07:24:16.168 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T07:24:16.168 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.log_readahead_size: 0 2024-09-19T07:24:16.168 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T07:24:16.168 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T07:24:16.168 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T07:24:16.169 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T07:24:16.169 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T07:24:16.169 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T07:24:16.169 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T07:24:16.169 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_background_jobs: 2 2024-09-19T07:24:16.169 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_background_compactions: -1 2024-09-19T07:24:16.169 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_subcompactions: 1 2024-09-19T07:24:16.169 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T07:24:16.169 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T07:24:16.169 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T07:24:16.169 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T07:24:16.169 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T07:24:16.169 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T07:24:16.170 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T07:24:16.170 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T07:24:16.170 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_open_files: -1 2024-09-19T07:24:16.170 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T07:24:16.170 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T07:24:16.170 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T07:24:16.170 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T07:24:16.170 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_background_flushes: -1 2024-09-19T07:24:16.170 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Compression algorithms supported: 2024-09-19T07:24:16.170 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: kZSTD supported: 0 2024-09-19T07:24:16.170 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: kXpressCompression supported: 0 2024-09-19T07:24:16.170 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: kBZip2Compression supported: 0 2024-09-19T07:24:16.170 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T07:24:16.170 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: kLZ4Compression supported: 1 2024-09-19T07:24:16.170 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: kZlibCompression supported: 1 2024-09-19T07:24:16.170 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T07:24:16.171 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: kSnappyCompression supported: 1 2024-09-19T07:24:16.171 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T07:24:16.171 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T07:24:16.171 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-19T07:24:16.171 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T07:24:16.171 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T07:24:16.171 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.merge_operator: 2024-09-19T07:24:16.171 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compaction_filter: None 2024-09-19T07:24:16.171 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T07:24:16.171 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T07:24:16.171 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T07:24:16.171 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T07:24:16.171 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556256613320) 2024-09-19T07:24:16.171 INFO:journalctl@ceph.mon.b.smithi189.stdout: cache_index_and_filter_blocks: 1 2024-09-19T07:24:16.172 INFO:journalctl@ceph.mon.b.smithi189.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T07:24:16.172 INFO:journalctl@ceph.mon.b.smithi189.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T07:24:16.172 INFO:journalctl@ceph.mon.b.smithi189.stdout: pin_top_level_index_and_filter: 1 2024-09-19T07:24:16.172 INFO:journalctl@ceph.mon.b.smithi189.stdout: index_type: 0 2024-09-19T07:24:16.172 INFO:journalctl@ceph.mon.b.smithi189.stdout: data_block_index_type: 0 2024-09-19T07:24:16.172 INFO:journalctl@ceph.mon.b.smithi189.stdout: index_shortening: 1 2024-09-19T07:24:16.172 INFO:journalctl@ceph.mon.b.smithi189.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T07:24:16.172 INFO:journalctl@ceph.mon.b.smithi189.stdout: checksum: 4 2024-09-19T07:24:16.172 INFO:journalctl@ceph.mon.b.smithi189.stdout: no_block_cache: 0 2024-09-19T07:24:16.172 INFO:journalctl@ceph.mon.b.smithi189.stdout: block_cache: 0x5562566071f0 2024-09-19T07:24:16.172 INFO:journalctl@ceph.mon.b.smithi189.stdout: block_cache_name: BinnedLRUCache 2024-09-19T07:24:16.172 INFO:journalctl@ceph.mon.b.smithi189.stdout: block_cache_options: 2024-09-19T07:24:16.173 INFO:journalctl@ceph.mon.b.smithi189.stdout: capacity : 536870912 2024-09-19T07:24:16.173 INFO:journalctl@ceph.mon.b.smithi189.stdout: num_shard_bits : 4 2024-09-19T07:24:16.173 INFO:journalctl@ceph.mon.b.smithi189.stdout: strict_capacity_limit : 0 2024-09-19T07:24:16.173 INFO:journalctl@ceph.mon.b.smithi189.stdout: high_pri_pool_ratio: 0.000 2024-09-19T07:24:16.173 INFO:journalctl@ceph.mon.b.smithi189.stdout: block_cache_compressed: (nil) 2024-09-19T07:24:16.173 INFO:journalctl@ceph.mon.b.smithi189.stdout: persistent_cache: (nil) 2024-09-19T07:24:16.173 INFO:journalctl@ceph.mon.b.smithi189.stdout: block_size: 4096 2024-09-19T07:24:16.173 INFO:journalctl@ceph.mon.b.smithi189.stdout: block_size_deviation: 10 2024-09-19T07:24:16.173 INFO:journalctl@ceph.mon.b.smithi189.stdout: block_restart_interval: 16 2024-09-19T07:24:16.173 INFO:journalctl@ceph.mon.b.smithi189.stdout: index_block_restart_interval: 1 2024-09-19T07:24:16.173 INFO:journalctl@ceph.mon.b.smithi189.stdout: metadata_block_size: 4096 2024-09-19T07:24:16.173 INFO:journalctl@ceph.mon.b.smithi189.stdout: partition_filters: 0 2024-09-19T07:24:16.173 INFO:journalctl@ceph.mon.b.smithi189.stdout: use_delta_encoding: 1 2024-09-19T07:24:16.173 INFO:journalctl@ceph.mon.b.smithi189.stdout: filter_policy: bloomfilter 2024-09-19T07:24:16.174 INFO:journalctl@ceph.mon.b.smithi189.stdout: whole_key_filtering: 1 2024-09-19T07:24:16.174 INFO:journalctl@ceph.mon.b.smithi189.stdout: verify_compression: 0 2024-09-19T07:24:16.174 INFO:journalctl@ceph.mon.b.smithi189.stdout: read_amp_bytes_per_bit: 0 2024-09-19T07:24:16.174 INFO:journalctl@ceph.mon.b.smithi189.stdout: format_version: 5 2024-09-19T07:24:16.174 INFO:journalctl@ceph.mon.b.smithi189.stdout: enable_index_compression: 1 2024-09-19T07:24:16.174 INFO:journalctl@ceph.mon.b.smithi189.stdout: block_align: 0 2024-09-19T07:24:16.174 INFO:journalctl@ceph.mon.b.smithi189.stdout: max_auto_readahead_size: 262144 2024-09-19T07:24:16.174 INFO:journalctl@ceph.mon.b.smithi189.stdout: prepopulate_block_cache: 0 2024-09-19T07:24:16.175 INFO:journalctl@ceph.mon.b.smithi189.stdout: initial_auto_readahead_size: 8192 2024-09-19T07:24:16.175 INFO:journalctl@ceph.mon.b.smithi189.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T07:24:16.175 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T07:24:16.175 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T07:24:16.175 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compression: NoCompression 2024-09-19T07:24:16.175 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T07:24:16.175 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T07:24:16.175 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T07:24:16.175 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.num_levels: 7 2024-09-19T07:24:16.176 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T07:24:16.176 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T07:24:16.176 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T07:24:16.176 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T07:24:16.176 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T07:24:16.176 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T07:24:16.176 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T07:24:16.176 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T07:24:16.176 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T07:24:16.176 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T07:24:16.176 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T07:24:16.176 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T07:24:16.177 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T07:24:16.177 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T07:24:16.177 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T07:24:16.177 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T07:24:16.177 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T07:24:16.177 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T07:24:16.177 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T07:24:16.177 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T07:24:16.178 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T07:24:16.178 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T07:24:16.178 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T07:24:16.178 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T07:24:16.178 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T07:24:16.178 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T07:24:16.178 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T07:24:16.178 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T07:24:16.178 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T07:24:16.178 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T07:24:16.178 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T07:24:16.178 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T07:24:16.178 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T07:24:16.178 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T07:24:16.179 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T07:24:16.179 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T07:24:16.179 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T07:24:16.179 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T07:24:16.179 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T07:24:16.179 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T07:24:16.179 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T07:24:16.179 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T07:24:16.179 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T07:24:16.180 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T07:24:16.180 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T07:24:16.180 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T07:24:16.180 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T07:24:16.180 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T07:24:16.180 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T07:24:16.180 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T07:24:16.180 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T07:24:16.180 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T07:24:16.180 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T07:24:16.180 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T07:24:16.181 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.inplace_update_support: 0 2024-09-19T07:24:16.181 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T07:24:16.181 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T07:24:16.181 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T07:24:16.181 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T07:24:16.181 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.bloom_locality: 0 2024-09-19T07:24:16.181 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.max_successive_merges: 0 2024-09-19T07:24:16.181 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T07:24:16.181 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T07:24:16.181 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T07:24:16.181 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T07:24:16.182 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.ttl: 2592000 2024-09-19T07:24:16.182 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T07:24:16.182 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T07:24:16.182 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T07:24:16.182 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.enable_blob_files: false 2024-09-19T07:24:16.182 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.min_blob_size: 0 2024-09-19T07:24:16.182 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T07:24:16.182 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T07:24:16.183 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T07:24:16.183 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T07:24:16.183 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T07:24:16.183 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T07:24:16.183 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T07:24:16.183 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T07:24:16.184 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-19T07:24:16.184 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-19T07:24:16.184 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 02def615-a12b-40fa-aed0-8b1428f8a2ca 2024-09-19T07:24:16.184 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726730656025963, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-19T07:24:16.184 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-19T07:24:16.184 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726730656026362, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726730656, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "02def615-a12b-40fa-aed0-8b1428f8a2ca", "db_session_id": "6WJBA84RX07GRJUMH32L", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-19T07:24:16.184 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726730656026457, "job": 1, "event": "recovery_finished"} 2024-09-19T07:24:16.185 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-19T07:24:16.185 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T07:24:16.185 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556256634e00 2024-09-19T07:24:16.185 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: DB pointer 0x556256730000 2024-09-19T07:24:16.185 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T07:24:16.185 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T07:24:16.186 INFO:journalctl@ceph.mon.b.smithi189.stdout: ** DB Stats ** 2024-09-19T07:24:16.186 INFO:journalctl@ceph.mon.b.smithi189.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T07:24:16.186 INFO:journalctl@ceph.mon.b.smithi189.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T07:24:16.186 INFO:journalctl@ceph.mon.b.smithi189.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T07:24:16.186 INFO:journalctl@ceph.mon.b.smithi189.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T07:24:16.186 INFO:journalctl@ceph.mon.b.smithi189.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T07:24:16.186 INFO:journalctl@ceph.mon.b.smithi189.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T07:24:16.186 INFO:journalctl@ceph.mon.b.smithi189.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T07:24:16.187 INFO:journalctl@ceph.mon.b.smithi189.stdout: 2024-09-19T07:24:16.187 INFO:journalctl@ceph.mon.b.smithi189.stdout: ** Compaction Stats [default] ** 2024-09-19T07:24:16.187 INFO:journalctl@ceph.mon.b.smithi189.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T07:24:16.187 INFO:journalctl@ceph.mon.b.smithi189.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T07:24:16.187 INFO:journalctl@ceph.mon.b.smithi189.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T07:24:16.187 INFO:journalctl@ceph.mon.b.smithi189.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T07:24:16.187 INFO:journalctl@ceph.mon.b.smithi189.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T07:24:16.187 INFO:journalctl@ceph.mon.b.smithi189.stdout: 2024-09-19T07:24:16.188 INFO:journalctl@ceph.mon.b.smithi189.stdout: ** Compaction Stats [default] ** 2024-09-19T07:24:16.188 INFO:journalctl@ceph.mon.b.smithi189.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T07:24:16.188 INFO:journalctl@ceph.mon.b.smithi189.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T07:24:16.188 INFO:journalctl@ceph.mon.b.smithi189.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T07:24:16.188 INFO:journalctl@ceph.mon.b.smithi189.stdout: 2024-09-19T07:24:16.188 INFO:journalctl@ceph.mon.b.smithi189.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T07:24:16.188 INFO:journalctl@ceph.mon.b.smithi189.stdout: 2024-09-19T07:24:16.188 INFO:journalctl@ceph.mon.b.smithi189.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T07:24:16.188 INFO:journalctl@ceph.mon.b.smithi189.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T07:24:16.189 INFO:journalctl@ceph.mon.b.smithi189.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T07:24:16.189 INFO:journalctl@ceph.mon.b.smithi189.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T07:24:16.189 INFO:journalctl@ceph.mon.b.smithi189.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T07:24:16.189 INFO:journalctl@ceph.mon.b.smithi189.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T07:24:16.189 INFO:journalctl@ceph.mon.b.smithi189.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T07:24:16.189 INFO:journalctl@ceph.mon.b.smithi189.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T07:24:16.189 INFO:journalctl@ceph.mon.b.smithi189.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T07:24:16.189 INFO:journalctl@ceph.mon.b.smithi189.stdout: Block cache BinnedLRUCache@0x5562566071f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-19T07:24:16.189 INFO:journalctl@ceph.mon.b.smithi189.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-19T07:24:16.189 INFO:journalctl@ceph.mon.b.smithi189.stdout: 2024-09-19T07:24:16.189 INFO:journalctl@ceph.mon.b.smithi189.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T07:24:16.189 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-19T07:24:16.189 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: using public_addr v2:172.21.15.189:0/0 -> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2024-09-19T07:24:16.190 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: starting mon.b rank -1 at public addrs [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] at bind addrs [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:24:16.190 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.b@-1(???) e0 preinit fsid c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:24:16.190 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.b@-1(synchronizing).mds e1 new map 2024-09-19T07:24:16.190 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-19T07:24:16.190 INFO:journalctl@ceph.mon.b.smithi189.stdout: e1 2024-09-19T07:24:16.190 INFO:journalctl@ceph.mon.b.smithi189.stdout: btime 2024-09-19T07:22:32:059587+0000 2024-09-19T07:24:16.190 INFO:journalctl@ceph.mon.b.smithi189.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T07:24:16.190 INFO:journalctl@ceph.mon.b.smithi189.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T07:24:16.190 INFO:journalctl@ceph.mon.b.smithi189.stdout: legacy client fscid: -1 2024-09-19T07:24:16.190 INFO:journalctl@ceph.mon.b.smithi189.stdout: 2024-09-19T07:24:16.190 INFO:journalctl@ceph.mon.b.smithi189.stdout: No filesystems configured 2024-09-19T07:24:16.190 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T07:24:16.191 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T07:24:16.191 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T07:24:16.191 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T07:24:16.191 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T07:24:16.191 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T07:24:16.191 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T07:24:16.191 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T07:24:16.191 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T07:24:16.191 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T07:24:16.191 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:24:16.192 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.192 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.192 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.192 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.192 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:24:16.192 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:16.193 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:16.193 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-19T07:24:16.193 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: Updating smithi189:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:24:16.193 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:24:16.193 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: Updating smithi189:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:24:16.193 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: Deploying cephadm binary to smithi203 2024-09-19T07:24:16.194 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:16.194 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.194 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.194 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.194 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:16.194 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.194 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:16.195 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.195 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.195 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: Added host smithi203 2024-09-19T07:24:16.195 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.195 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:16.195 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:24:16.196 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/4098429054' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T07:24:16.196 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.196 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.196 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.197 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.197 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:24:16.197 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:16.197 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:16.197 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:16.197 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-19T07:24:16.198 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/4098429054' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T07:24:16.198 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:24:16.198 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.198 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.198 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.198 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: Updating smithi203:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:24:16.199 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:24:16.199 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: Updating smithi203:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:24:16.199 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.199 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:16.199 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:16.199 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:16.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:24:16.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:16.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi159:172.21.15.159=a;smithi189:172.21.15.189=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:24:16.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: Saving service mon spec with placement smithi159:172.21.15.159=a;smithi189:172.21.15.189=b;smithi203:172.21.15.203=c;count:3 2024-09-19T07:24:16.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: Deploying daemon mon.c on smithi203 2024-09-19T07:24:16.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:16.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='client.? 172.21.15.203:0/1642955359' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:24:16.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:16.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:24:16.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:16.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.a calling monitor election 2024-09-19T07:24:16.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:16.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:16.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:16.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.c calling monitor election 2024-09-19T07:24:16.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:16.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:16.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:16.201 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:16.202 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T07:24:16.202 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: monmap epoch 2 2024-09-19T07:24:16.202 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: fsid c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:24:16.202 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: last_changed 2024-09-19T07:24:07.174125+0000 2024-09-19T07:24:16.202 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: created 2024-09-19T07:22:28.312056+0000 2024-09-19T07:24:16.202 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: min_mon_release 19 (squid) 2024-09-19T07:24:16.202 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: election_strategy: 1 2024-09-19T07:24:16.202 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: 0: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.a 2024-09-19T07:24:16.202 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-09-19T07:24:16.202 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: fsmap 2024-09-19T07:24:16.202 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:24:16.202 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mgrmap e13: x(active, since 41s) 2024-09-19T07:24:16.202 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: overall HEALTH_OK 2024-09-19T07:24:16.203 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.203 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.203 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:16.203 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:24:16.203 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:16.203 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: Deploying daemon mon.b on smithi189 2024-09-19T07:24:16.203 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='client.? 172.21.15.203:0/1130573377' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:24:16.203 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:16.203 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:16.203 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:16.203 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: from='client.? 172.21.15.203:0/2643022009' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:24:16.203 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 ceph-mon[34851]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T07:24:16.204 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 bash[34833]: e3b690676dfcb4ef321000fdd584730a6dda23ed638567563faa90e0dc3e0714 2024-09-19T07:24:16.204 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:16 smithi189 systemd[1]: Started Ceph mon.b for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:24:16.556 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T07:24:16.557 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T07:24:16.810 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:24:21.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:24:21.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: mon.a calling monitor election 2024-09-19T07:24:21.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: mon.c calling monitor election 2024-09-19T07:24:21.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:21.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:21.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: mon.b calling monitor election 2024-09-19T07:24:21.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:21.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.389 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:21.389 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.389 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T07:24:21.389 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: monmap epoch 3 2024-09-19T07:24:21.389 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: fsid c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:24:21.389 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: last_changed 2024-09-19T07:24:16.051516+0000 2024-09-19T07:24:21.389 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: created 2024-09-19T07:22:28.312056+0000 2024-09-19T07:24:21.389 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: min_mon_release 19 (squid) 2024-09-19T07:24:21.389 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: election_strategy: 1 2024-09-19T07:24:21.389 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: 0: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.a 2024-09-19T07:24:21.389 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-09-19T07:24:21.389 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: 2: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.b 2024-09-19T07:24:21.389 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: fsmap 2024-09-19T07:24:21.389 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:24:21.390 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: mgrmap e13: x(active, since 50s) 2024-09-19T07:24:21.390 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: overall HEALTH_OK 2024-09-19T07:24:21.390 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:21.390 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:21.390 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:21.390 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:21.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:24:21.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: mon.a calling monitor election 2024-09-19T07:24:21.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: mon.c calling monitor election 2024-09-19T07:24:21.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:21.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:21.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: mon.b calling monitor election 2024-09-19T07:24:21.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:21.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:21.538 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.538 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T07:24:21.538 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: monmap epoch 3 2024-09-19T07:24:21.538 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: fsid c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:24:21.538 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: last_changed 2024-09-19T07:24:16.051516+0000 2024-09-19T07:24:21.538 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: created 2024-09-19T07:22:28.312056+0000 2024-09-19T07:24:21.538 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: min_mon_release 19 (squid) 2024-09-19T07:24:21.538 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: election_strategy: 1 2024-09-19T07:24:21.538 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: 0: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.a 2024-09-19T07:24:21.538 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-09-19T07:24:21.538 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: 2: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.b 2024-09-19T07:24:21.539 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: fsmap 2024-09-19T07:24:21.539 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:24:21.539 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: mgrmap e13: x(active, since 50s) 2024-09-19T07:24:21.539 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: overall HEALTH_OK 2024-09-19T07:24:21.539 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:21.539 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:21.539 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:21.539 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:21.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:24:21.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: mon.a calling monitor election 2024-09-19T07:24:21.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: mon.c calling monitor election 2024-09-19T07:24:21.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:24:21.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:21.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: mon.b calling monitor election 2024-09-19T07:24:21.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:21.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:21.558 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:21.558 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T07:24:21.558 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: monmap epoch 3 2024-09-19T07:24:21.558 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: fsid c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:24:21.558 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: last_changed 2024-09-19T07:24:16.051516+0000 2024-09-19T07:24:21.558 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: created 2024-09-19T07:22:28.312056+0000 2024-09-19T07:24:21.558 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: min_mon_release 19 (squid) 2024-09-19T07:24:21.558 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: election_strategy: 1 2024-09-19T07:24:21.558 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: 0: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.a 2024-09-19T07:24:21.558 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-09-19T07:24:21.559 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: 2: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.b 2024-09-19T07:24:21.559 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: fsmap 2024-09-19T07:24:21.559 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:24:21.559 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: mgrmap e13: x(active, since 50s) 2024-09-19T07:24:21.559 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: overall HEALTH_OK 2024-09-19T07:24:21.559 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:21.559 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:21.559 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:21.559 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:21.649 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.646+0000 7f8f467f9640 1 -- 172.21.15.203:0/1345561241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f40102820 msgr2=0x7f8f28005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:21.649 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.646+0000 7f8f467f9640 1 --2- 172.21.15.203:0/1345561241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f40102820 0x7f8f28005580 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f30012c60 tx=0x7f8f3002c620 comp rx=0 tx=0).stop 2024-09-19T07:24:21.650 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.646+0000 7f8f467f9640 1 -- 172.21.15.203:0/1345561241 shutdown_connections 2024-09-19T07:24:21.650 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.646+0000 7f8f467f9640 1 --2- 172.21.15.203:0/1345561241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f40102820 0x7f8f28005580 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:21.650 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.646+0000 7f8f467f9640 1 -- 172.21.15.203:0/1345561241 >> 172.21.15.203:0/1345561241 conn(0x7f8f400fc220 msgr2=0x7f8f400fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:21.650 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.646+0000 7f8f467f9640 1 -- 172.21.15.203:0/1345561241 shutdown_connections 2024-09-19T07:24:21.650 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.646+0000 7f8f467f9640 1 -- 172.21.15.203:0/1345561241 wait complete. 2024-09-19T07:24:21.650 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.647+0000 7f8f467f9640 1 Processor -- start 2024-09-19T07:24:21.650 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.647+0000 7f8f467f9640 1 -- start start 2024-09-19T07:24:21.650 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.647+0000 7f8f467f9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f40102820 0x7f8f40100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:21.650 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.647+0000 7f8f467f9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f401009c0 0x7f8f40100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:21.650 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.647+0000 7f8f467f9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f401a34c0 0x7f8f401a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:21.651 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.647+0000 7f8f467f9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8f40109780 con 0x7f8f401009c0 2024-09-19T07:24:21.651 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.647+0000 7f8f467f9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8f40109600 con 0x7f8f401a34c0 2024-09-19T07:24:21.651 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.647+0000 7f8f467f9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8f40109900 con 0x7f8f40102820 2024-09-19T07:24:21.651 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f3ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f40102820 0x7f8f40100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:21.651 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f3ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f40102820 0x7f8f40100480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:45522/0 (socket says 172.21.15.203:45522) 2024-09-19T07:24:21.651 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f3ffff640 1 -- 172.21.15.203:0/988871648 learned_addr learned my addr 172.21.15.203:0/988871648 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-19T07:24:21.651 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f3ffff640 1 -- 172.21.15.203:0/988871648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f40102820 msgr2=0x7f8f40100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T07:24:21.651 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f3ffff640 1 -- 172.21.15.203:0/988871648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f40102820 msgr2=0x7f8f40100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T07:24:21.651 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f3ffff640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f40102820 0x7f8f40100480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T07:24:21.651 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f3f7fe640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f401009c0 0x7f8f40100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:21.651 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f3ffff640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f40102820 0x7f8f40100480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:24:21.651 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f44d96640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f401a34c0 0x7f8f401a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:21.651 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f44d96640 1 -- 172.21.15.203:0/988871648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f401a34c0 msgr2=0x7f8f401a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-19T07:24:21.652 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f44d96640 1 -- 172.21.15.203:0/988871648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f401a34c0 msgr2=0x7f8f401a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T07:24:21.652 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f44d96640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f401a34c0 0x7f8f401a58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T07:24:21.652 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f44d96640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f401a34c0 0x7f8f401a58b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:24:21.652 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f3f7fe640 1 -- 172.21.15.203:0/988871648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f40102820 msgr2=0x7f8f40100480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:24:21.652 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f3f7fe640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f40102820 0x7f8f40100480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:21.652 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f3f7fe640 1 -- 172.21.15.203:0/988871648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f401a34c0 msgr2=0x7f8f401a58b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:24:21.652 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f3f7fe640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f401a34c0 0x7f8f401a58b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:21.652 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f3f7fe640 1 -- 172.21.15.203:0/988871648 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f401a5ef0 con 0x7f8f401009c0 2024-09-19T07:24:21.652 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.648+0000 7f8f3f7fe640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f401009c0 0x7f8f40100e20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400b450 tx=0x7f8f3400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:21.652 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.649+0000 7f8f3d7fa640 1 -- 172.21.15.203:0/988871648 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34004230 con 0x7f8f401009c0 2024-09-19T07:24:21.652 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.649+0000 7f8f467f9640 1 -- 172.21.15.203:0/988871648 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f401a61e0 con 0x7f8f401009c0 2024-09-19T07:24:21.652 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.649+0000 7f8f3d7fa640 1 -- 172.21.15.203:0/988871648 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f340043d0 con 0x7f8f401009c0 2024-09-19T07:24:21.652 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.649+0000 7f8f3d7fa640 1 -- 172.21.15.203:0/988871648 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34003cb0 con 0x7f8f401009c0 2024-09-19T07:24:21.652 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.649+0000 7f8f467f9640 1 -- 172.21.15.203:0/988871648 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f401a6720 con 0x7f8f401009c0 2024-09-19T07:24:21.653 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.649+0000 7f8f3d7fa640 1 -- 172.21.15.203:0/988871648 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f8f3400f050 con 0x7f8f401009c0 2024-09-19T07:24:21.653 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.650+0000 7f8f467f9640 1 -- 172.21.15.203:0/988871648 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f4010a890 con 0x7f8f401009c0 2024-09-19T07:24:21.653 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.650+0000 7f8f3d7fa640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8f2c039730 0x7f8f2c03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:21.654 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.650+0000 7f8f3d7fa640 1 -- 172.21.15.203:0/988871648 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8f3404c150 con 0x7f8f401009c0 2024-09-19T07:24:21.654 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.650+0000 7f8f3ffff640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8f2c039730 0x7f8f2c03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:21.654 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.651+0000 7f8f3ffff640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8f2c039730 0x7f8f2c03bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8f30012aa0 tx=0x7f8f30005de0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:21.655 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.655+0000 7f8f3d7fa640 1 -- 172.21.15.203:0/988871648 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8f3400d040 con 0x7f8f401009c0 2024-09-19T07:24:21.906 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.905+0000 7f8f467f9640 1 -- 172.21.15.203:0/988871648 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8f40190930 con 0x7f8f401009c0 2024-09-19T07:24:21.906 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.905+0000 7f8f3d7fa640 1 -- 172.21.15.203:0/988871648 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1289 (secure 0 0 0) 0x7f8f3400d540 con 0x7f8f401009c0 2024-09-19T07:24:21.906 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:24:21.906 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":3,"fsid":"c9d18048-7657-11ef-bceb-c7b262605968","modified":"2024-09-19T07:24:16.051516Z","created":"2024-09-19T07:22:28.312056Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-19T07:24:21.906 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 3 2024-09-19T07:24:21.908 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.908+0000 7f8f467f9640 1 -- 172.21.15.203:0/988871648 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8f2c039730 msgr2=0x7f8f2c03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:21.908 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.908+0000 7f8f467f9640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8f2c039730 0x7f8f2c03bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8f30012aa0 tx=0x7f8f30005de0 comp rx=0 tx=0).stop 2024-09-19T07:24:21.908 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.908+0000 7f8f467f9640 1 -- 172.21.15.203:0/988871648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f401009c0 msgr2=0x7f8f40100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:21.908 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.908+0000 7f8f467f9640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f401009c0 0x7f8f40100e20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400b450 tx=0x7f8f3400b910 comp rx=0 tx=0).stop 2024-09-19T07:24:21.908 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.908+0000 7f8f467f9640 1 -- 172.21.15.203:0/988871648 shutdown_connections 2024-09-19T07:24:21.908 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.908+0000 7f8f467f9640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8f2c039730 0x7f8f2c03bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:21.909 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.908+0000 7f8f467f9640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f401a34c0 0x7f8f401a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:21.909 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.908+0000 7f8f467f9640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f401009c0 0x7f8f40100e20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:21.909 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.908+0000 7f8f467f9640 1 --2- 172.21.15.203:0/988871648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f40102820 0x7f8f40100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:21.909 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.908+0000 7f8f467f9640 1 -- 172.21.15.203:0/988871648 >> 172.21.15.203:0/988871648 conn(0x7f8f400fc220 msgr2=0x7f8f400fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:21.909 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.908+0000 7f8f467f9640 1 -- 172.21.15.203:0/988871648 shutdown_connections 2024-09-19T07:24:21.909 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:21.908+0000 7f8f467f9640 1 -- 172.21.15.203:0/988871648 wait complete. 2024-09-19T07:24:22.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:22 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:22.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:22 smithi203 ceph-mon[35335]: from='client.? 172.21.15.203:0/988871648' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:24:22.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:22 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:22.238 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:22 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:22.238 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:22 smithi189 ceph-mon[34851]: from='client.? 172.21.15.203:0/988871648' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:24:22.238 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:22 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:22.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:22 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:22.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:22 smithi159 ceph-mon[30472]: from='client.? 172.21.15.203:0/988871648' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:24:22.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:22 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:24:22.515 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T07:24:22.516 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T07:24:22.889 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:24:23.293 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:23 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:23.293 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:23 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:23.293 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:23 smithi159 ceph-mon[30472]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:23.293 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:23 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:23.293 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:23 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:23.293 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:23 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:23.293 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:23 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:23.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.533+0000 7ffb85400640 1 -- 172.21.15.159:0/1844032751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb801097c0 msgr2=0x7ffb80109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:23.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.533+0000 7ffb85400640 1 --2- 172.21.15.159:0/1844032751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb801097c0 0x7ffb80109ba0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffb70009880 tx=0x7ffb7002a5d0 comp rx=0 tx=0).stop 2024-09-19T07:24:23.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.533+0000 7ffb85400640 1 -- 172.21.15.159:0/1844032751 shutdown_connections 2024-09-19T07:24:23.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.533+0000 7ffb85400640 1 --2- 172.21.15.159:0/1844032751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb801097c0 0x7ffb80109ba0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:23.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.533+0000 7ffb85400640 1 -- 172.21.15.159:0/1844032751 >> 172.21.15.159:0/1844032751 conn(0x7ffb800ff4e0 msgr2=0x7ffb80101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:23.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.533+0000 7ffb85400640 1 -- 172.21.15.159:0/1844032751 shutdown_connections 2024-09-19T07:24:23.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.533+0000 7ffb85400640 1 -- 172.21.15.159:0/1844032751 wait complete. 2024-09-19T07:24:23.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.534+0000 7ffb85400640 1 Processor -- start 2024-09-19T07:24:23.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.534+0000 7ffb85400640 1 -- start start 2024-09-19T07:24:23.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.535+0000 7ffb85400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb801097c0 0x7ffb80199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:23.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.535+0000 7ffb85400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb80199ac0 0x7ffb8019de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:23.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.535+0000 7ffb85400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb8019e390 0x7ffb8019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:23.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.535+0000 7ffb85400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8010de20 con 0x7ffb80199ac0 2024-09-19T07:24:23.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.535+0000 7ffb85400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8010dca0 con 0x7ffb801097c0 2024-09-19T07:24:23.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.535+0000 7ffb85400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8010dfa0 con 0x7ffb8019e390 2024-09-19T07:24:23.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:23 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:23.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:23 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:23.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:23 smithi203 ceph-mon[35335]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:23.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:23 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:23.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:23 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:23.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:23 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:23.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:23 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:23.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.535+0000 7ffb7e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb801097c0 0x7ffb80199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:23.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.535+0000 7ffb7de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb80199ac0 0x7ffb8019de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:23.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.535+0000 7ffb7e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb801097c0 0x7ffb80199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:37022/0 (socket says 172.21.15.159:37022) 2024-09-19T07:24:23.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.535+0000 7ffb7e800640 1 -- 172.21.15.159:0/862204503 learned_addr learned my addr 172.21.15.159:0/862204503 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:24:23.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.536+0000 7ffb7e800640 1 -- 172.21.15.159:0/862204503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb8019e390 msgr2=0x7ffb8019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:23.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.536+0000 7ffb7f200640 1 --2- 172.21.15.159:0/862204503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb8019e390 0x7ffb8019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:23.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.536+0000 7ffb7e800640 1 --2- 172.21.15.159:0/862204503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb8019e390 0x7ffb8019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:23.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.536+0000 7ffb7e800640 1 -- 172.21.15.159:0/862204503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb80199ac0 msgr2=0x7ffb8019de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:23.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.536+0000 7ffb7e800640 1 --2- 172.21.15.159:0/862204503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb80199ac0 0x7ffb8019de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:23.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.536+0000 7ffb7e800640 1 -- 172.21.15.159:0/862204503 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb801a6100 con 0x7ffb801097c0 2024-09-19T07:24:23.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.536+0000 7ffb7f200640 1 --2- 172.21.15.159:0/862204503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb8019e390 0x7ffb8019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:24:23.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.536+0000 7ffb7e800640 1 --2- 172.21.15.159:0/862204503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb801097c0 0x7ffb80199580 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffb70031550 tx=0x7ffb700042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:23.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.536+0000 7ffb77400640 1 -- 172.21.15.159:0/862204503 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7003f030 con 0x7ffb801097c0 2024-09-19T07:24:23.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.536+0000 7ffb77400640 1 -- 172.21.15.159:0/862204503 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb70003ce0 con 0x7ffb801097c0 2024-09-19T07:24:23.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.536+0000 7ffb77400640 1 -- 172.21.15.159:0/862204503 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb70035070 con 0x7ffb801097c0 2024-09-19T07:24:23.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.537+0000 7ffb85400640 1 -- 172.21.15.159:0/862204503 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb801a62d0 con 0x7ffb801097c0 2024-09-19T07:24:23.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.537+0000 7ffb85400640 1 -- 172.21.15.159:0/862204503 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb801a6790 con 0x7ffb801097c0 2024-09-19T07:24:23.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.538+0000 7ffb85400640 1 -- 172.21.15.159:0/862204503 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb80109c20 con 0x7ffb801097c0 2024-09-19T07:24:23.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.538+0000 7ffb77400640 1 -- 172.21.15.159:0/862204503 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7ffb70042030 con 0x7ffb801097c0 2024-09-19T07:24:23.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.539+0000 7ffb77400640 1 --2- 172.21.15.159:0/862204503 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ffb64039730 0x7ffb6403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:23.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.539+0000 7ffb77400640 1 -- 172.21.15.159:0/862204503 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ffb7006d090 con 0x7ffb801097c0 2024-09-19T07:24:23.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.539+0000 7ffb7de00640 1 --2- 172.21.15.159:0/862204503 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ffb64039730 0x7ffb6403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:23.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.540+0000 7ffb7de00640 1 --2- 172.21.15.159:0/862204503 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ffb64039730 0x7ffb6403bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009a30 tx=0x7ffb6c009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:23.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.544+0000 7ffb77400640 1 -- 172.21.15.159:0/862204503 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ffb7003bce0 con 0x7ffb801097c0 2024-09-19T07:24:23.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:23 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:23.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:23 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:23.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:23 smithi189 ceph-mon[34851]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:23.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:23 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:23.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:23 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:23.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:23 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:23.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:23 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:23.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.738+0000 7ffb85400640 1 -- 172.21.15.159:0/862204503 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ffb801a6b20 con 0x7ffb801097c0 2024-09-19T07:24:23.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.739+0000 7ffb77400640 1 -- 172.21.15.159:0/862204503 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7ffb700433c0 con 0x7ffb801097c0 2024-09-19T07:24:23.740 INFO:teuthology.orchestra.run.smithi159.stdout:# minimal ceph.conf for c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:24:23.740 INFO:teuthology.orchestra.run.smithi159.stdout:[global] 2024-09-19T07:24:23.740 INFO:teuthology.orchestra.run.smithi159.stdout: fsid = c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:24:23.740 INFO:teuthology.orchestra.run.smithi159.stdout: mon_host = [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] 2024-09-19T07:24:23.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.741+0000 7ffb85400640 1 -- 172.21.15.159:0/862204503 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ffb64039730 msgr2=0x7ffb6403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:23.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.741+0000 7ffb85400640 1 --2- 172.21.15.159:0/862204503 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ffb64039730 0x7ffb6403bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009a30 tx=0x7ffb6c009190 comp rx=0 tx=0).stop 2024-09-19T07:24:23.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.741+0000 7ffb85400640 1 -- 172.21.15.159:0/862204503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb801097c0 msgr2=0x7ffb80199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:23.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.741+0000 7ffb85400640 1 --2- 172.21.15.159:0/862204503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb801097c0 0x7ffb80199580 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffb70031550 tx=0x7ffb700042f0 comp rx=0 tx=0).stop 2024-09-19T07:24:23.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.742+0000 7ffb85400640 1 -- 172.21.15.159:0/862204503 shutdown_connections 2024-09-19T07:24:23.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.742+0000 7ffb85400640 1 --2- 172.21.15.159:0/862204503 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ffb64039730 0x7ffb6403bbf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:23.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.742+0000 7ffb85400640 1 --2- 172.21.15.159:0/862204503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb8019e390 0x7ffb8019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:23.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.742+0000 7ffb85400640 1 --2- 172.21.15.159:0/862204503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb80199ac0 0x7ffb8019de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:23.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.742+0000 7ffb85400640 1 --2- 172.21.15.159:0/862204503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb801097c0 0x7ffb80199580 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:23.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.742+0000 7ffb85400640 1 -- 172.21.15.159:0/862204503 >> 172.21.15.159:0/862204503 conn(0x7ffb800ff4e0 msgr2=0x7ffb801072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:23.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.742+0000 7ffb85400640 1 -- 172.21.15.159:0/862204503 shutdown_connections 2024-09-19T07:24:23.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:23.742+0000 7ffb85400640 1 -- 172.21.15.159:0/862204503 wait complete. 2024-09-19T07:24:24.535 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T07:24:24.535 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:24:24.535 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T07:24:24.596 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:24:24.596 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:24:24.676 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-19T07:24:24.676 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-19T07:24:24.676 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-19T07:24:24.676 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: Updating smithi203:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:24:24.676 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: Updating smithi189:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:24:24.676 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.676 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.676 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.676 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.676 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: Updating smithi159:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:24:24.677 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/862204503' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:24.677 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.677 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.677 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.677 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.677 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.677 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.677 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.677 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:24:24.677 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:24:24.677 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:24 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:24.693 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-19T07:24:24.693 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T07:24:24.721 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-19T07:24:24.721 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-19T07:24:24.721 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-19T07:24:24.721 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: Updating smithi203:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:24:24.721 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: Updating smithi189:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:24:24.721 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.721 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.721 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.722 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.722 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: Updating smithi159:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:24:24.722 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/862204503' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:24.722 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.722 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.722 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.722 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.722 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.722 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.722 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.722 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:24:24.722 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:24:24.722 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:24 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:24.726 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-19T07:24:24.726 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:24:24.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-19T07:24:24.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-19T07:24:24.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: Updating smithi203:/etc/ceph/ceph.conf 2024-09-19T07:24:24.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: Updating smithi203:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:24:24.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: Updating smithi189:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:24:24.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: Updating smithi159:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:24:24.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/862204503' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:24.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.788 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:24.788 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:24:24.788 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:24:24.788 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:24 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:24.795 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-19T07:24:24.795 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T07:24:24.826 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-19T07:24:24.826 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:24:24.894 INFO:tasks.cephadm:Adding mgr.x on smithi159 2024-09-19T07:24:24.894 INFO:tasks.cephadm:Adding mgr.y on smithi189 2024-09-19T07:24:24.894 INFO:tasks.cephadm:Adding mgr.z on smithi203 2024-09-19T07:24:24.894 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi159=x;smithi189=y;smithi203=z' 2024-09-19T07:24:25.175 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:24:25.607 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:25 smithi203 ceph-mon[35335]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T07:24:25.607 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:25 smithi203 ceph-mon[35335]: Reconfiguring daemon mon.a on smithi159 2024-09-19T07:24:25.608 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:25 smithi203 ceph-mon[35335]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:25.615 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.606+0000 7f2bc18c4640 1 -- 172.21.15.203:0/2656000419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bbc1005f0 msgr2=0x7f2bbc1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:25.615 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.606+0000 7f2bc18c4640 1 --2- 172.21.15.203:0/2656000419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bbc1005f0 0x7f2bbc1009f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009880 tx=0x7f2bac02a5d0 comp rx=0 tx=0).stop 2024-09-19T07:24:25.615 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.607+0000 7f2bc18c4640 1 -- 172.21.15.203:0/2656000419 shutdown_connections 2024-09-19T07:24:25.615 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.607+0000 7f2bc18c4640 1 --2- 172.21.15.203:0/2656000419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bbc1005f0 0x7f2bbc1009f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:25.615 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.607+0000 7f2bc18c4640 1 -- 172.21.15.203:0/2656000419 >> 172.21.15.203:0/2656000419 conn(0x7f2bbc0fbda0 msgr2=0x7f2bbc0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:25.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.607+0000 7f2bc18c4640 1 -- 172.21.15.203:0/2656000419 shutdown_connections 2024-09-19T07:24:25.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.607+0000 7f2bc18c4640 1 -- 172.21.15.203:0/2656000419 wait complete. 2024-09-19T07:24:25.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.607+0000 7f2bc18c4640 1 Processor -- start 2024-09-19T07:24:25.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.607+0000 7f2bc18c4640 1 -- start start 2024-09-19T07:24:25.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bc18c4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bbc1005f0 0x7f2bbc197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:25.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bc18c4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bbc197560 0x7f2bbc19c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:25.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bc18c4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bbc1979e0 0x7f2bbc19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:25.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bc18c4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc1066e0 con 0x7f2bbc1005f0 2024-09-19T07:24:25.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bc18c4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc106560 con 0x7f2bbc1979e0 2024-09-19T07:24:25.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bc18c4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc106860 con 0x7f2bbc197560 2024-09-19T07:24:25.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bba7fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bbc197560 0x7f2bbc19c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:25.616 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bba7fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bbc197560 0x7f2bbc19c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:45532/0 (socket says 172.21.15.203:45532) 2024-09-19T07:24:25.617 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bba7fc640 1 -- 172.21.15.203:0/31263967 learned_addr learned my addr 172.21.15.203:0/31263967 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-19T07:24:25.617 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bbaffd640 1 --2- 172.21.15.203:0/31263967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bbc1005f0 0x7f2bbc197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:25.617 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bbb7fe640 1 --2- 172.21.15.203:0/31263967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bbc1979e0 0x7f2bbc19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:25.617 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bba7fc640 1 -- 172.21.15.203:0/31263967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bbc1979e0 msgr2=0x7f2bbc19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:25.617 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bba7fc640 1 --2- 172.21.15.203:0/31263967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bbc1979e0 0x7f2bbc19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:25.617 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bba7fc640 1 -- 172.21.15.203:0/31263967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bbc1005f0 msgr2=0x7f2bbc197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:25.617 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bba7fc640 1 --2- 172.21.15.203:0/31263967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bbc1005f0 0x7f2bbc197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:25.617 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bba7fc640 1 -- 172.21.15.203:0/31263967 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc19f420 con 0x7f2bbc197560 2024-09-19T07:24:25.617 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bbaffd640 1 --2- 172.21.15.203:0/31263967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bbc1005f0 0x7f2bbc197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:24:25.617 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bbb7fe640 1 --2- 172.21.15.203:0/31263967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bbc1979e0 0x7f2bbc19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:24:25.617 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bba7fc640 1 --2- 172.21.15.203:0/31263967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bbc197560 0x7f2bbc19c5d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000b520 tx=0x7f2bb000b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:25.621 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bc08c2640 1 -- 172.21.15.203:0/31263967 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0004230 con 0x7f2bbc197560 2024-09-19T07:24:25.622 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.608+0000 7f2bc08c2640 1 -- 172.21.15.203:0/31263967 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bb00043d0 con 0x7f2bbc197560 2024-09-19T07:24:25.622 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.609+0000 7f2bc08c2640 1 -- 172.21.15.203:0/31263967 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0003d60 con 0x7f2bbc197560 2024-09-19T07:24:25.622 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.609+0000 7f2bc18c4640 1 -- 172.21.15.203:0/31263967 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bbc19f710 con 0x7f2bbc197560 2024-09-19T07:24:25.622 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.609+0000 7f2bc18c4640 1 -- 172.21.15.203:0/31263967 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bbc19fcd0 con 0x7f2bbc197560 2024-09-19T07:24:25.622 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.609+0000 7f2bc08c2640 1 -- 172.21.15.203:0/31263967 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f2bb000e070 con 0x7f2bbc197560 2024-09-19T07:24:25.622 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.609+0000 7f2bc08c2640 1 --2- 172.21.15.203:0/31263967 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2ba4039730 0x7f2ba403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:25.622 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.610+0000 7f2bc08c2640 1 -- 172.21.15.203:0/31263967 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2bb004bd40 con 0x7f2bbc197560 2024-09-19T07:24:25.622 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.610+0000 7f2bc18c4640 1 -- 172.21.15.203:0/31263967 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bbc19ff90 con 0x7f2bbc197560 2024-09-19T07:24:25.622 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.610+0000 7f2bbaffd640 1 --2- 172.21.15.203:0/31263967 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2ba4039730 0x7f2ba403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:25.622 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.610+0000 7f2bbaffd640 1 --2- 172.21.15.203:0/31263967 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2ba4039730 0x7f2ba403bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2bac031440 tx=0x7f2bac0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:25.622 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.614+0000 7f2bc08c2640 1 -- 172.21.15.203:0/31263967 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2bb0014020 con 0x7f2bbc197560 2024-09-19T07:24:25.794 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.793+0000 7f2bc18c4640 1 -- 172.21.15.203:0/31263967 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi159=x;smithi189=y;smithi203=z", "target": ["mon-mgr", ""]}) -- 0x7f2bbc062630 con 0x7f2ba4039730 2024-09-19T07:24:25.799 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.799+0000 7f2bc08c2640 1 -- 172.21.15.203:0/31263967 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2bbc062630 con 0x7f2ba4039730 2024-09-19T07:24:25.800 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mgr update... 2024-09-19T07:24:25.802 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.801+0000 7f2bc18c4640 1 -- 172.21.15.203:0/31263967 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2ba4039730 msgr2=0x7f2ba403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:25.802 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.801+0000 7f2bc18c4640 1 --2- 172.21.15.203:0/31263967 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2ba4039730 0x7f2ba403bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2bac031440 tx=0x7f2bac0058d0 comp rx=0 tx=0).stop 2024-09-19T07:24:25.802 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.801+0000 7f2bc18c4640 1 -- 172.21.15.203:0/31263967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bbc197560 msgr2=0x7f2bbc19c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:25.802 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.801+0000 7f2bc18c4640 1 --2- 172.21.15.203:0/31263967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bbc197560 0x7f2bbc19c5d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000b520 tx=0x7f2bb000b9e0 comp rx=0 tx=0).stop 2024-09-19T07:24:25.802 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.801+0000 7f2bc18c4640 1 -- 172.21.15.203:0/31263967 shutdown_connections 2024-09-19T07:24:25.802 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.801+0000 7f2bc18c4640 1 --2- 172.21.15.203:0/31263967 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2ba4039730 0x7f2ba403bbf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:25.803 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.801+0000 7f2bc18c4640 1 --2- 172.21.15.203:0/31263967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bbc1979e0 0x7f2bbc19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:25.803 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.801+0000 7f2bc18c4640 1 --2- 172.21.15.203:0/31263967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bbc197560 0x7f2bbc19c5d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:25.803 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.801+0000 7f2bc18c4640 1 --2- 172.21.15.203:0/31263967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bbc1005f0 0x7f2bbc197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:25.803 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.802+0000 7f2bc18c4640 1 -- 172.21.15.203:0/31263967 >> 172.21.15.203:0/31263967 conn(0x7f2bbc0fbda0 msgr2=0x7f2bbc06acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:25.803 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.802+0000 7f2bc18c4640 1 -- 172.21.15.203:0/31263967 shutdown_connections 2024-09-19T07:24:25.803 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:24:25.802+0000 7f2bc18c4640 1 -- 172.21.15.203:0/31263967 wait complete. 2024-09-19T07:24:25.804 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:25 smithi159 ceph-mon[30472]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T07:24:25.804 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:25 smithi159 ceph-mon[30472]: Reconfiguring daemon mon.a on smithi159 2024-09-19T07:24:25.804 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:25 smithi159 ceph-mon[30472]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:25.805 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:25 smithi189 ceph-mon[34851]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T07:24:25.805 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:25 smithi189 ceph-mon[34851]: Reconfiguring daemon mon.a on smithi159 2024-09-19T07:24:25.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:25 smithi189 ceph-mon[34851]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:26.447 DEBUG:teuthology.orchestra.run.smithi189:mgr.y> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mgr.y.service 2024-09-19T07:24:26.450 DEBUG:teuthology.orchestra.run.smithi203:mgr.z> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mgr.z.service 2024-09-19T07:24:26.453 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T07:24:26.453 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:24:26.453 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T07:24:26.488 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T07:24:26.489 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_1 2024-09-19T07:24:26.565 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T07:24:26.566 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:24:26.566 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-09-19T07:24:26.566 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:24:26.566 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:24:26.566 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-19 07:23:44.102179110 +0000 2024-09-19T07:24:26.566 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-19 07:19:02.047981435 +0000 2024-09-19T07:24:26.566 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-19 07:19:02.047981435 +0000 2024-09-19T07:24:26.566 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-19 07:19:02.047981435 +0000 2024-09-19T07:24:26.567 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T07:24:26.662 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-19T07:24:26.662 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-19T07:24:26.662 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000302609 s, 1.7 MB/s 2024-09-19T07:24:26.665 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T07:24:26.702 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_2 2024-09-19T07:24:26.780 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T07:24:26.780 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:24:26.780 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-09-19T07:24:26.780 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:24:26.780 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:24:26.780 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-19 07:23:44.152178179 +0000 2024-09-19T07:24:26.780 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-19 07:19:02.486973329 +0000 2024-09-19T07:24:26.780 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-19 07:19:02.486973329 +0000 2024-09-19T07:24:26.780 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-19 07:19:02.486973329 +0000 2024-09-19T07:24:26.781 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T07:24:26.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:26 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:26.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:26 smithi203 ceph-mon[35335]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi159=x;smithi189=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:24:26.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:26 smithi203 ceph-mon[35335]: Saving service mgr spec with placement smithi159=x;smithi189=y;smithi203=z;count:3 2024-09-19T07:24:26.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:26 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:26.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:26 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:26.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:26 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:26.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:26 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:24:26.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:26 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:24:26.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:26 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:26.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:26 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:26.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:26 smithi189 ceph-mon[34851]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi159=x;smithi189=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:24:26.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:26 smithi189 ceph-mon[34851]: Saving service mgr spec with placement smithi159=x;smithi189=y;smithi203=z;count:3 2024-09-19T07:24:26.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:26 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:26.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:26 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:26.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:26 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:26.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:26 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:24:26.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:26 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:24:26.807 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:26 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:26.873 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-19T07:24:26.873 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-19T07:24:26.873 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000265508 s, 1.9 MB/s 2024-09-19T07:24:26.875 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T07:24:26.913 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_3 2024-09-19T07:24:26.988 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T07:24:26.988 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:24:26.988 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-09-19T07:24:26.988 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:24:26.988 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:24:26.988 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-19 07:23:44.202177248 +0000 2024-09-19T07:24:26.988 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-19 07:19:02.945964855 +0000 2024-09-19T07:24:26.988 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-19 07:19:02.945964855 +0000 2024-09-19T07:24:26.988 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-19 07:19:02.945964855 +0000 2024-09-19T07:24:26.989 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T07:24:27.079 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:26 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:27.079 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:26 smithi159 ceph-mon[30472]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi159=x;smithi189=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:24:27.079 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:26 smithi159 ceph-mon[30472]: Saving service mgr spec with placement smithi159=x;smithi189=y;smithi203=z;count:3 2024-09-19T07:24:27.079 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:26 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:27.079 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:26 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:27.079 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:26 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:27.079 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:26 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:24:27.079 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:26 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:24:27.079 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:26 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:27.085 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-19T07:24:27.085 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-19T07:24:27.085 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000266698 s, 1.9 MB/s 2024-09-19T07:24:27.088 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T07:24:27.124 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_4 2024-09-19T07:24:27.200 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T07:24:27.201 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:24:27.201 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-09-19T07:24:27.201 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:24:27.201 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:24:27.201 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-19 07:23:44.256176243 +0000 2024-09-19T07:24:27.201 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-19 07:19:03.409956288 +0000 2024-09-19T07:24:27.201 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-19 07:19:03.409956288 +0000 2024-09-19T07:24:27.201 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-19 07:19:03.409956288 +0000 2024-09-19T07:24:27.201 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T07:24:27.293 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-19T07:24:27.294 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-19T07:24:27.294 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000291799 s, 1.8 MB/s 2024-09-19T07:24:27.296 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T07:24:27.333 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-19T07:24:27.333 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T07:24:27.353 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T07:24:27.353 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_1 2024-09-19T07:24:27.411 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T07:24:27.412 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:24:27.412 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-19T07:24:27.412 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:24:27.412 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:24:27.412 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-19 07:24:15.504516947 +0000 2024-09-19T07:24:27.412 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-19 07:18:34.367347104 +0000 2024-09-19T07:24:27.412 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-19 07:18:34.367347104 +0000 2024-09-19T07:24:27.412 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-19 07:18:34.367347104 +0000 2024-09-19T07:24:27.412 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T07:24:27.480 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-19T07:24:27.480 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-19T07:24:27.480 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000224907 s, 2.3 MB/s 2024-09-19T07:24:27.481 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T07:24:27.540 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_2 2024-09-19T07:24:27.597 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T07:24:27.598 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:24:27.598 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-19T07:24:27.598 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:24:27.598 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:24:27.598 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-19 07:24:15.504516947 +0000 2024-09-19T07:24:27.598 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-19 07:18:34.686340656 +0000 2024-09-19T07:24:27.598 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-19 07:18:34.686340656 +0000 2024-09-19T07:24:27.598 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-19 07:18:34.686340656 +0000 2024-09-19T07:24:27.598 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T07:24:27.665 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-19T07:24:27.666 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-19T07:24:27.666 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000184766 s, 2.8 MB/s 2024-09-19T07:24:27.667 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T07:24:27.724 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_3 2024-09-19T07:24:27.780 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T07:24:27.780 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:24:27.780 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-19T07:24:27.781 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:24:27.781 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:24:27.781 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-19 07:24:15.505516928 +0000 2024-09-19T07:24:27.781 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-19 07:18:35.002334268 +0000 2024-09-19T07:24:27.781 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-19 07:18:35.002334268 +0000 2024-09-19T07:24:27.781 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-19 07:18:35.002334268 +0000 2024-09-19T07:24:27.782 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T07:24:27.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:27 smithi189 ceph-mon[34851]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T07:24:27.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:27 smithi189 ceph-mon[34851]: Reconfiguring daemon mgr.x on smithi159 2024-09-19T07:24:27.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:27 smithi189 ceph-mon[34851]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:27.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:27.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:27.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:24:27.807 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:24:27.807 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:27.830 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-19T07:24:27.830 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-19T07:24:27.830 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000223798 s, 2.3 MB/s 2024-09-19T07:24:27.831 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T07:24:27.889 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_4 2024-09-19T07:24:27.945 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T07:24:27.945 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:24:27.945 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-19T07:24:27.945 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:24:27.945 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:24:27.945 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-19 07:24:15.505516928 +0000 2024-09-19T07:24:27.945 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-19 07:18:35.322327800 +0000 2024-09-19T07:24:27.946 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-19 07:18:35.322327800 +0000 2024-09-19T07:24:27.946 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-19 07:18:35.322327800 +0000 2024-09-19T07:24:27.946 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T07:24:28.008 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-19T07:24:28.009 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-19T07:24:28.009 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000210993 s, 2.4 MB/s 2024-09-19T07:24:28.010 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T07:24:28.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:27 smithi203 ceph-mon[35335]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T07:24:28.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:27 smithi203 ceph-mon[35335]: Reconfiguring daemon mgr.x on smithi159 2024-09-19T07:24:28.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:27 smithi203 ceph-mon[35335]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:28.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:24:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:24:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:28.066 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-09-19T07:24:28.067 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T07:24:28.086 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T07:24:28.086 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 2024-09-19T07:24:28.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:27 smithi159 ceph-mon[30472]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T07:24:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:27 smithi159 ceph-mon[30472]: Reconfiguring daemon mgr.x on smithi159 2024-09-19T07:24:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:27 smithi159 ceph-mon[30472]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:24:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:24:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:28.144 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T07:24:28.144 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:24:28.144 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-19T07:24:28.144 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:24:28.145 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:24:28.145 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-19 07:24:06.628969072 +0000 2024-09-19T07:24:28.145 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-19 07:18:38.972990653 +0000 2024-09-19T07:24:28.145 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-19 07:18:38.972990653 +0000 2024-09-19T07:24:28.145 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-19 07:18:38.972990653 +0000 2024-09-19T07:24:28.145 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T07:24:28.212 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-19T07:24:28.213 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-19T07:24:28.213 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000287111 s, 1.8 MB/s 2024-09-19T07:24:28.214 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T07:24:28.273 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 2024-09-19T07:24:28.331 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T07:24:28.331 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:24:28.331 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-19T07:24:28.331 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:24:28.331 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:24:28.331 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-19 07:24:06.628969072 +0000 2024-09-19T07:24:28.331 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-19 07:18:39.310982125 +0000 2024-09-19T07:24:28.331 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-19 07:18:39.310982125 +0000 2024-09-19T07:24:28.331 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-19 07:18:39.310982125 +0000 2024-09-19T07:24:28.332 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T07:24:28.397 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-19T07:24:28.398 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-19T07:24:28.398 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.00020516 s, 2.5 MB/s 2024-09-19T07:24:28.399 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T07:24:28.457 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 2024-09-19T07:24:28.513 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T07:24:28.514 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:24:28.514 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-19T07:24:28.514 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:24:28.514 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:24:28.514 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-19 07:24:06.629969047 +0000 2024-09-19T07:24:28.514 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-19 07:18:39.644973699 +0000 2024-09-19T07:24:28.514 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-19 07:18:39.644973699 +0000 2024-09-19T07:24:28.514 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-19 07:18:39.644973699 +0000 2024-09-19T07:24:28.514 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T07:24:28.579 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-19T07:24:28.579 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-19T07:24:28.580 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.00021348 s, 2.4 MB/s 2024-09-19T07:24:28.581 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T07:24:28.638 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 2024-09-19T07:24:28.694 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T07:24:28.694 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:24:28.694 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-19T07:24:28.694 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:24:28.694 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:24:28.694 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-09-19 07:24:06.629969047 +0000 2024-09-19T07:24:28.695 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-09-19 07:18:39.966965574 +0000 2024-09-19T07:24:28.695 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-09-19 07:18:39.966965574 +0000 2024-09-19T07:24:28.695 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-09-19 07:18:39.966965574 +0000 2024-09-19T07:24:28.695 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T07:24:28.758 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-09-19T07:24:28.758 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-09-19T07:24:28.759 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000227816 s, 2.2 MB/s 2024-09-19T07:24:28.760 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T07:24:28.816 INFO:tasks.cephadm:Deploying osd.0 on smithi159 with /dev/vg_nvme/lv_4... 2024-09-19T07:24:28.817 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T07:24:28.906 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:28 smithi189 ceph-mon[34851]: Reconfiguring mon.b (monmap changed)... 2024-09-19T07:24:28.906 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:28 smithi189 ceph-mon[34851]: Reconfiguring daemon mon.b on smithi189 2024-09-19T07:24:29.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:28 smithi203 ceph-mon[35335]: Reconfiguring mon.b (monmap changed)... 2024-09-19T07:24:29.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:28 smithi203 ceph-mon[35335]: Reconfiguring daemon mon.b on smithi189 2024-09-19T07:24:29.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:28 smithi159 ceph-mon[30472]: Reconfiguring mon.b (monmap changed)... 2024-09-19T07:24:29.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:28 smithi159 ceph-mon[30472]: Reconfiguring daemon mon.b on smithi189 2024-09-19T07:24:29.179 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:24:29.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:29 smithi159 ceph-mon[30472]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:29.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:29 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:29.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:29 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:29.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:29 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:24:29.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:29 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:24:29.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:29 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:30.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:29 smithi203 ceph-mon[35335]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:30.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:29 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:30.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:29 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:30.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:29 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:24:30.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:29 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:24:30.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:29 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:30.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:29 smithi189 ceph-mon[34851]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:30.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:29 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:30.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:29 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:30.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:29 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:24:30.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:29 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:24:30.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:29 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:30.709 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:30 smithi159 ceph-mon[30472]: Reconfiguring mon.c (monmap changed)... 2024-09-19T07:24:30.710 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:30 smithi159 ceph-mon[30472]: Reconfiguring daemon mon.c on smithi203 2024-09-19T07:24:30.710 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:30.710 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:30.710 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:30.710 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:30.710 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:30.710 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:30.710 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:24:30.710 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T07:24:30.710 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:24:30.710 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:30.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:30 smithi203 ceph-mon[35335]: Reconfiguring mon.c (monmap changed)... 2024-09-19T07:24:30.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:30 smithi203 ceph-mon[35335]: Reconfiguring daemon mon.c on smithi203 2024-09-19T07:24:30.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:30.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:30.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:30.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:30.913 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:30.914 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:30.914 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:24:30.914 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T07:24:30.914 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:24:30.914 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:30 smithi189 ceph-mon[34851]: Reconfiguring mon.c (monmap changed)... 2024-09-19T07:24:31.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:30 smithi189 ceph-mon[34851]: Reconfiguring daemon mon.c on smithi203 2024-09-19T07:24:31.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:31.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:31.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:31.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:31.058 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:31.058 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:31.058 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:24:31.058 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T07:24:31.058 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:24:31.059 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:31.916 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:31 smithi159 ceph-mon[30472]: Deploying daemon mgr.z on smithi203 2024-09-19T07:24:31.917 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:31 smithi159 ceph-mon[30472]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:31.937 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:31 smithi203 ceph-mon[35335]: Deploying daemon mgr.z on smithi203 2024-09-19T07:24:31.937 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:31 smithi203 ceph-mon[35335]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:32.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:31 smithi189 ceph-mon[34851]: Deploying daemon mgr.z on smithi203 2024-09-19T07:24:32.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:31 smithi189 ceph-mon[34851]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:32.198 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:24:32.230 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi159:vg_nvme/lv_4 2024-09-19T07:24:32.536 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:32 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:32.498+0000 7f0a5a12e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T07:24:32.594 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:24:32.914 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:32 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:32.557+0000 7f0a5a12e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T07:24:32.914 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:32 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:32.704+0000 7f0a5a12e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T07:24:32.915 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:32 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:32.769+0000 7f0a5a12e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T07:24:32.915 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:32 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:32.913+0000 7f0a5a12e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:24:33.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.130+0000 7f9bc7e00640 1 -- 172.21.15.159:0/669511179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bc010e1e0 msgr2=0x7f9bc01105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:33.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.130+0000 7f9bc7e00640 1 --2- 172.21.15.159:0/669511179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bc010e1e0 0x7f9bc01105d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009be0 tx=0x7f9bb802a6d0 comp rx=0 tx=0).stop 2024-09-19T07:24:33.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.131+0000 7f9bc7e00640 1 -- 172.21.15.159:0/669511179 shutdown_connections 2024-09-19T07:24:33.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.131+0000 7f9bc7e00640 1 --2- 172.21.15.159:0/669511179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bc010e1e0 0x7f9bc01105d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:33.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.131+0000 7f9bc7e00640 1 --2- 172.21.15.159:0/669511179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bc0074c50 0x7f9bc00750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:33.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.131+0000 7f9bc7e00640 1 --2- 172.21.15.159:0/669511179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bc0076220 0x7f9bc0074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:33.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.131+0000 7f9bc7e00640 1 -- 172.21.15.159:0/669511179 >> 172.21.15.159:0/669511179 conn(0x7f9bc00fd390 msgr2=0x7f9bc00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:33.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.131+0000 7f9bc7e00640 1 -- 172.21.15.159:0/669511179 shutdown_connections 2024-09-19T07:24:33.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.131+0000 7f9bc7e00640 1 -- 172.21.15.159:0/669511179 wait complete. 2024-09-19T07:24:33.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.132+0000 7f9bc7e00640 1 Processor -- start 2024-09-19T07:24:33.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.132+0000 7f9bc7e00640 1 -- start start 2024-09-19T07:24:33.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.132+0000 7f9bc7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bc0074c50 0x7f9bc019e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:33.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.132+0000 7f9bc7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bc0076220 0x7f9bc019e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:33.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.132+0000 7f9bc7e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bc010e1e0 0x7f9bc01a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:33.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.132+0000 7f9bc7e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc0113ea0 con 0x7f9bc0074c50 2024-09-19T07:24:33.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.132+0000 7f9bc7e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc0113d20 con 0x7f9bc0076220 2024-09-19T07:24:33.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.132+0000 7f9bc7e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc0114020 con 0x7f9bc010e1e0 2024-09-19T07:24:33.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.132+0000 7f9bc4a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bc0076220 0x7f9bc019e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:33.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.132+0000 7f9bc5e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bc010e1e0 0x7f9bc01a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:33.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.132+0000 7f9bc5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bc0074c50 0x7f9bc019e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:33.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.132+0000 7f9bc4a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bc0076220 0x7f9bc019e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:41914/0 (socket says 172.21.15.159:41914) 2024-09-19T07:24:33.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.132+0000 7f9bc5e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bc010e1e0 0x7f9bc01a5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:52456/0 (socket says 172.21.15.159:52456) 2024-09-19T07:24:33.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.132+0000 7f9bc4a00640 1 -- 172.21.15.159:0/689393071 learned_addr learned my addr 172.21.15.159:0/689393071 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:24:33.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.133+0000 7f9bc4a00640 1 -- 172.21.15.159:0/689393071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bc010e1e0 msgr2=0x7f9bc01a5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:33.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.133+0000 7f9bc4a00640 1 --2- 172.21.15.159:0/689393071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bc010e1e0 0x7f9bc01a5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:33.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.133+0000 7f9bc4a00640 1 -- 172.21.15.159:0/689393071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bc0074c50 msgr2=0x7f9bc019e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:33.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.133+0000 7f9bc4a00640 1 --2- 172.21.15.159:0/689393071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bc0074c50 0x7f9bc019e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:33.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.133+0000 7f9bc4a00640 1 -- 172.21.15.159:0/689393071 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc01a60b0 con 0x7f9bc0076220 2024-09-19T07:24:33.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.133+0000 7f9bc4a00640 1 --2- 172.21.15.159:0/689393071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bc0076220 0x7f9bc019e9c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00e9c0 tx=0x7f9bac00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:33.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.133+0000 7f9bbe000640 1 -- 172.21.15.159:0/689393071 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac00cde0 con 0x7f9bc0076220 2024-09-19T07:24:33.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.133+0000 7f9bc7e00640 1 -- 172.21.15.159:0/689393071 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc01a63a0 con 0x7f9bc0076220 2024-09-19T07:24:33.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.133+0000 7f9bbe000640 1 -- 172.21.15.159:0/689393071 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bac0035f0 con 0x7f9bc0076220 2024-09-19T07:24:33.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.133+0000 7f9bbe000640 1 -- 172.21.15.159:0/689393071 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac010650 con 0x7f9bc0076220 2024-09-19T07:24:33.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.133+0000 7f9bc7e00640 1 -- 172.21.15.159:0/689393071 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc01a6930 con 0x7f9bc0076220 2024-09-19T07:24:33.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.135+0000 7f9bbe000640 1 -- 172.21.15.159:0/689393071 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f9bac0037a0 con 0x7f9bc0076220 2024-09-19T07:24:33.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.135+0000 7f9bc7e00640 1 -- 172.21.15.159:0/689393071 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc0104290 con 0x7f9bc0076220 2024-09-19T07:24:33.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.135+0000 7f9bbe000640 1 --2- 172.21.15.159:0/689393071 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9ba8039500 0x7f9ba803b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:24:33.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.135+0000 7f9bbe000640 1 -- 172.21.15.159:0/689393071 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9bac04ca70 con 0x7f9bc0076220 2024-09-19T07:24:33.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.135+0000 7f9bc5400640 1 --2- 172.21.15.159:0/689393071 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9ba8039500 0x7f9ba803b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:24:33.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.136+0000 7f9bc5400640 1 --2- 172.21.15.159:0/689393071 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9ba8039500 0x7f9ba803b9c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40095f0 tx=0x7f9bb4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:24:33.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.140+0000 7f9bbe000640 1 -- 172.21.15.159:0/689393071 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9bac00a250 con 0x7f9bc0076220 2024-09-19T07:24:33.286 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:33 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:33.054+0000 7f0a5a12e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T07:24:33.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:33.324+0000 7f9bc7e00640 1 -- 172.21.15.159:0/689393071 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9bc0062630 con 0x7f9ba8039500 2024-09-19T07:24:33.619 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:33 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:33.367+0000 7f0a5a12e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T07:24:33.619 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:33 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:33.428+0000 7f0a5a12e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T07:24:33.619 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:33.619 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:33.619 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:33.619 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:24:33.619 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T07:24:33.620 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:24:33.620 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:33.620 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:33 smithi203 ceph-mon[35335]: Deploying daemon mgr.y on smithi189 2024-09-19T07:24:33.620 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:33 smithi203 ceph-mon[35335]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:33.620 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:24:33.620 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:24:33.620 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:33.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:33.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:33.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:33.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:24:33.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T07:24:33.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:24:33.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:33.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:33 smithi159 ceph-mon[30472]: Deploying daemon mgr.y on smithi189 2024-09-19T07:24:33.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:33 smithi159 ceph-mon[30472]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:33.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:24:33.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:24:33.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:33.796 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:33.796 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:33.796 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:33.796 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:24:33.796 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T07:24:33.796 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:24:33.796 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:33.796 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:33 smithi189 ceph-mon[34851]: Deploying daemon mgr.y on smithi189 2024-09-19T07:24:33.796 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:33 smithi189 ceph-mon[34851]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:33.797 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:24:33.797 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:24:33.797 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:33.987 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:33 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:33.618+0000 7f0a5a12e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T07:24:34.456 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:34 smithi189 systemd[1]: Starting Ceph mgr.y for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:24:34.457 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:34 smithi189 podman[36202]: 2024-09-19 07:24:34.456159746 +0000 UTC m=+0.093006402 container create 315155a80885975d05fd80fe26702a2b33acb6d8530ac4eabf9fbc868155cc93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T07:24:34.741 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:34 smithi189 ceph-mon[34851]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:24:34.741 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:34 smithi189 podman[36202]: 2024-09-19 07:24:34.378766336 +0000 UTC m=+0.015612999 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:24:34.742 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:34 smithi189 podman[36202]: 2024-09-19 07:24:34.658645659 +0000 UTC m=+0.295492324 container init 315155a80885975d05fd80fe26702a2b33acb6d8530ac4eabf9fbc868155cc93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T07:24:34.742 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:34 smithi189 podman[36202]: 2024-09-19 07:24:34.662064668 +0000 UTC m=+0.298911327 container start 315155a80885975d05fd80fe26702a2b33acb6d8530ac4eabf9fbc868155cc93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T07:24:34.742 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:34 smithi189 ceph-mgr[36221]: -- 172.21.15.189:0/3739980690 <== mon.2 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f10f7250e0 con 0x55f10f703400 2024-09-19T07:24:34.772 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:34 smithi203 ceph-mon[35335]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:24:34.772 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:34 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:34.547+0000 7f0a5a12e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T07:24:34.772 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:34 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:34.617+0000 7f0a5a12e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T07:24:34.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:34 smithi159 ceph-mon[30472]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:24:34.999 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:34 smithi189 bash[36202]: 315155a80885975d05fd80fe26702a2b33acb6d8530ac4eabf9fbc868155cc93 2024-09-19T07:24:34.999 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:34 smithi189 systemd[1]: Started Ceph mgr.y for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:24:34.999 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:34 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:34.782+0000 7f5ba2fb9040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T07:24:34.999 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:34 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:34.843+0000 7f5ba2fb9040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T07:24:34.999 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:34 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:34.998+0000 7f5ba2fb9040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T07:24:35.306 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:35 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:35.068+0000 7f5ba2fb9040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T07:24:35.306 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:35 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:35.235+0000 7f5ba2fb9040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:24:35.671 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:35 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:35.402+0000 7f0a5a12e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T07:24:35.671 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:35 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:35.519+0000 7f0a5a12e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T07:24:35.672 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:35 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:35.670+0000 7f0a5a12e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:24:35.720 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:35 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:35.399+0000 7f5ba2fb9040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T07:24:35.953 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:35 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:35.733+0000 7f0a5a12e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T07:24:35.953 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:35 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:35.885+0000 7f0a5a12e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T07:24:35.953 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:35 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:35.953+0000 7f0a5a12e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T07:24:35.954 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:35 smithi203 ceph-mon[35335]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:35.954 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:35.954 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:35.954 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:35.954 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:35.954 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:35.954 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:35.981 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:35 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:35.732+0000 7f5ba2fb9040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T07:24:35.981 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:35 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:35.793+0000 7f5ba2fb9040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T07:24:35.981 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:35 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:35.980+0000 7f5ba2fb9040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T07:24:35.981 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:35 smithi189 ceph-mon[34851]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:35.982 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:35.982 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:35.982 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:35.982 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:35.982 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:35.982 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:36.084 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:35 smithi159 ceph-mon[30472]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:36.084 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:36.084 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:36.084 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:36.085 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:36.085 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:36.085 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:36.251 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:36 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:36.015+0000 7f0a5a12e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T07:24:36.251 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:36 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:36.181+0000 7f0a5a12e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T07:24:36.251 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:36 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:36.250+0000 7f0a5a12e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T07:24:36.523 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:36 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:36.318+0000 7f0a5a12e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T07:24:36.524 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:36 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:36.403+0000 7f0a5a12e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T07:24:36.524 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:36 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:36.463+0000 7f0a5a12e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T07:24:36.524 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:36 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:36.523+0000 7f0a5a12e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T07:24:37.306 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:36 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:36.885+0000 7f5ba2fb9040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T07:24:37.306 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:36 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:36.954+0000 7f5ba2fb9040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T07:24:37.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:37.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:37.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:37.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:37.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:37.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1666982202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "752f5466-fe04-408a-8566-86625947fd3b"}]: dispatch 2024-09-19T07:24:37.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1666982202' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "752f5466-fe04-408a-8566-86625947fd3b"}]': finished 2024-09-19T07:24:37.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-mon[34851]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T07:24:37.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:24:37.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-mon[34851]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:37.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:37.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:37.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:37.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:37.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:37.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:37 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1666982202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "752f5466-fe04-408a-8566-86625947fd3b"}]: dispatch 2024-09-19T07:24:37.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:37 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1666982202' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "752f5466-fe04-408a-8566-86625947fd3b"}]': finished 2024-09-19T07:24:37.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:37 smithi159 ceph-mon[30472]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T07:24:37.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:24:37.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:37 smithi159 ceph-mon[30472]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:37.536 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:24:37 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z[37137]: 2024-09-19T07:24:37.311+0000 7f0a5a12e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T07:24:37.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:37.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:37.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:37.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:37.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:37.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:37.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:37 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1666982202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "752f5466-fe04-408a-8566-86625947fd3b"}]: dispatch 2024-09-19T07:24:37.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:37 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1666982202' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "752f5466-fe04-408a-8566-86625947fd3b"}]': finished 2024-09-19T07:24:37.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:37 smithi203 ceph-mon[35335]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T07:24:37.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:24:37.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:37 smithi203 ceph-mon[35335]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:37.933 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:37.677+0000 7f5ba2fb9040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T07:24:37.933 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:37.792+0000 7f5ba2fb9040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T07:24:38.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:38.080+0000 7f9bbe000640 1 -- 172.21.15.159:0/689393071 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mgrmap(e 14) ==== 92409+0+0 (secure 0 0 0) 0x7f9bac021430 con 0x7f9bc0076220 2024-09-19T07:24:38.213 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:38 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/4179599044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:24:38.213 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:38 smithi189 ceph-mon[34851]: Standby manager daemon z started 2024-09-19T07:24:38.213 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:38 smithi189 ceph-mon[34851]: from='mgr.? 172.21.15.203:0/1151445279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T07:24:38.213 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:38 smithi189 ceph-mon[34851]: from='mgr.? 172.21.15.203:0/1151445279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:24:38.213 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:38 smithi189 ceph-mon[34851]: from='mgr.? 172.21.15.203:0/1151445279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T07:24:38.214 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:38 smithi189 ceph-mon[34851]: from='mgr.? 172.21.15.203:0/1151445279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:24:38.214 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:37.933+0000 7f5ba2fb9040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:24:38.214 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:37 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:37.995+0000 7f5ba2fb9040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T07:24:38.214 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:38 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:38.146+0000 7f5ba2fb9040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T07:24:38.214 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:38 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:38.212+0000 7f5ba2fb9040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T07:24:38.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:38 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/4179599044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:24:38.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:38 smithi159 ceph-mon[30472]: Standby manager daemon z started 2024-09-19T07:24:38.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:38 smithi159 ceph-mon[30472]: from='mgr.? 172.21.15.203:0/1151445279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T07:24:38.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:38 smithi159 ceph-mon[30472]: from='mgr.? 172.21.15.203:0/1151445279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:24:38.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:38 smithi159 ceph-mon[30472]: from='mgr.? 172.21.15.203:0/1151445279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T07:24:38.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:38 smithi159 ceph-mon[30472]: from='mgr.? 172.21.15.203:0/1151445279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:24:38.494 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:38 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:38.272+0000 7f5ba2fb9040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T07:24:38.494 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:38 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:38.429+0000 7f5ba2fb9040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T07:24:38.494 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:38 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:38.493+0000 7f5ba2fb9040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T07:24:38.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:38 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/4179599044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:24:38.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:38 smithi203 ceph-mon[35335]: Standby manager daemon z started 2024-09-19T07:24:38.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:38 smithi203 ceph-mon[35335]: from='mgr.? 172.21.15.203:0/1151445279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T07:24:38.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:38 smithi203 ceph-mon[35335]: from='mgr.? 172.21.15.203:0/1151445279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:24:38.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:38 smithi203 ceph-mon[35335]: from='mgr.? 172.21.15.203:0/1151445279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T07:24:38.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:38 smithi203 ceph-mon[35335]: from='mgr.? 172.21.15.203:0/1151445279' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:24:38.762 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:38 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:38.559+0000 7f5ba2fb9040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T07:24:38.762 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:38 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:38.643+0000 7f5ba2fb9040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T07:24:38.762 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:38 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:38.703+0000 7f5ba2fb9040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T07:24:38.762 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:38 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:38.761+0000 7f5ba2fb9040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T07:24:39.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:39 smithi159 ceph-mon[30472]: mgrmap e14: x(active, since 67s), standbys: z 2024-09-19T07:24:39.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:39 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T07:24:39.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:39 smithi159 ceph-mon[30472]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:39.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:39 smithi203 ceph-mon[35335]: mgrmap e14: x(active, since 67s), standbys: z 2024-09-19T07:24:39.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:39 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T07:24:39.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:39 smithi203 ceph-mon[35335]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:39.551 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:39 smithi189 ceph-mon[34851]: mgrmap e14: x(active, since 67s), standbys: z 2024-09-19T07:24:39.551 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:39 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T07:24:39.551 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:39 smithi189 ceph-mon[34851]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:39.806 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:24:39 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y[36217]: 2024-09-19T07:24:39.550+0000 7f5ba2fb9040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T07:24:40.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:40.098+0000 7f9bbe000640 1 -- 172.21.15.159:0/689393071 <== mon.2 v2:172.21.15.189:3300/0 8 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f9bac00f3c0 con 0x7f9bc0076220 2024-09-19T07:24:40.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:40 smithi159 ceph-mon[30472]: Standby manager daemon y started 2024-09-19T07:24:40.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:40 smithi159 ceph-mon[30472]: from='mgr.? 172.21.15.189:0/2059424246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T07:24:40.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:40 smithi159 ceph-mon[30472]: from='mgr.? 172.21.15.189:0/2059424246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:24:40.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:40 smithi159 ceph-mon[30472]: from='mgr.? 172.21.15.189:0/2059424246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T07:24:40.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:40 smithi159 ceph-mon[30472]: from='mgr.? 172.21.15.189:0/2059424246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:24:40.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:40 smithi203 ceph-mon[35335]: Standby manager daemon y started 2024-09-19T07:24:40.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:40 smithi203 ceph-mon[35335]: from='mgr.? 172.21.15.189:0/2059424246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T07:24:40.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:40 smithi203 ceph-mon[35335]: from='mgr.? 172.21.15.189:0/2059424246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:24:40.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:40 smithi203 ceph-mon[35335]: from='mgr.? 172.21.15.189:0/2059424246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T07:24:40.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:40 smithi203 ceph-mon[35335]: from='mgr.? 172.21.15.189:0/2059424246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:24:40.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:40 smithi189 ceph-mon[34851]: Standby manager daemon y started 2024-09-19T07:24:40.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:40 smithi189 ceph-mon[34851]: from='mgr.? 172.21.15.189:0/2059424246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T07:24:40.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:40 smithi189 ceph-mon[34851]: from='mgr.? 172.21.15.189:0/2059424246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:24:40.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:40 smithi189 ceph-mon[34851]: from='mgr.? 172.21.15.189:0/2059424246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T07:24:40.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:40 smithi189 ceph-mon[34851]: from='mgr.? 172.21.15.189:0/2059424246' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:24:41.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:41 smithi159 ceph-mon[30472]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-09-19T07:24:41.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:41 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T07:24:41.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:41 smithi159 ceph-mon[30472]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:41.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:41 smithi203 ceph-mon[35335]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-09-19T07:24:41.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:41 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T07:24:41.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:41 smithi203 ceph-mon[35335]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:41.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:41 smithi189 ceph-mon[34851]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-09-19T07:24:41.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:41 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T07:24:41.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:41 smithi189 ceph-mon[34851]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:43.706 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:43 smithi159 ceph-mon[30472]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:44.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:43 smithi203 ceph-mon[35335]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:44.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:43 smithi189 ceph-mon[34851]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:46.012 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:45 smithi159 ceph-mon[30472]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:46.012 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:45 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T07:24:46.012 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:45 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:46.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:45 smithi203 ceph-mon[35335]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:46.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:45 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T07:24:46.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:45 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:46.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:45 smithi189 ceph-mon[34851]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:46.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:45 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T07:24:46.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:45 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:47.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:46 smithi203 ceph-mon[35335]: Deploying daemon osd.0 on smithi159 2024-09-19T07:24:47.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:46 smithi189 ceph-mon[34851]: Deploying daemon osd.0 on smithi159 2024-09-19T07:24:47.118 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:46 smithi159 ceph-mon[30472]: Deploying daemon osd.0 on smithi159 2024-09-19T07:24:47.714 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:47 smithi159 ceph-mon[30472]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:48.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:47 smithi203 ceph-mon[35335]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:48.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:47 smithi189 ceph-mon[34851]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:49.855 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:49 smithi159 ceph-mon[30472]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:50.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:49 smithi203 ceph-mon[35335]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:50.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:49 smithi189 ceph-mon[34851]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:51.801 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:51 smithi159 ceph-mon[30472]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:52.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:51 smithi203 ceph-mon[35335]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:52.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:51 smithi189 ceph-mon[34851]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:54.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:53 smithi203 ceph-mon[35335]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:54.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:53 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:54.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:53 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:54.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:53 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:54.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:53 smithi189 ceph-mon[34851]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:54.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:53 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:54.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:53 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:54.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:53 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:54.065 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:53 smithi159 ceph-mon[30472]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:54.065 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:53 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:54.065 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:53 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:54.066 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:53 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:55.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:55 smithi159 ceph-mon[30472]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:56.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:55 smithi203 ceph-mon[35335]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:56.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:55 smithi189 ceph-mon[34851]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:57.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:57 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:57.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:57 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:57.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:57 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:57.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:57 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:57.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:57 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:57.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:57 smithi203 ceph-mon[35335]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:57.805 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:57 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:57.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:57 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:57.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:57 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:57.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:57 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:57.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:57 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:57.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:57 smithi189 ceph-mon[34851]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:57.840 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:57.840 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:57.840 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:24:57.840 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:24:57.840 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:57.840 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:57 smithi159 ceph-mon[30472]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:58.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:58.176+0000 7f9bbe000640 1 -- 172.21.15.159:0/689393071 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9bc0062630 con 0x7f9ba8039500 2024-09-19T07:24:58.177 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 0 on host 'smithi159' 2024-09-19T07:24:58.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:58.178+0000 7f9bc7e00640 1 -- 172.21.15.159:0/689393071 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9ba8039500 msgr2=0x7f9ba803b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:58.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:58.178+0000 7f9bc7e00640 1 --2- 172.21.15.159:0/689393071 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9ba8039500 0x7f9ba803b9c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40095f0 tx=0x7f9bb4005f90 comp rx=0 tx=0).stop 2024-09-19T07:24:58.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:58.178+0000 7f9bc7e00640 1 -- 172.21.15.159:0/689393071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bc0076220 msgr2=0x7f9bc019e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:24:58.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:58.178+0000 7f9bc7e00640 1 --2- 172.21.15.159:0/689393071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bc0076220 0x7f9bc019e9c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00e9c0 tx=0x7f9bac00ee80 comp rx=0 tx=0).stop 2024-09-19T07:24:58.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:58.179+0000 7f9bc7e00640 1 -- 172.21.15.159:0/689393071 shutdown_connections 2024-09-19T07:24:58.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:58.179+0000 7f9bc7e00640 1 --2- 172.21.15.159:0/689393071 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9ba8039500 0x7f9ba803b9c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:58.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:58.179+0000 7f9bc7e00640 1 --2- 172.21.15.159:0/689393071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bc010e1e0 0x7f9bc01a5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:58.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:58.179+0000 7f9bc7e00640 1 --2- 172.21.15.159:0/689393071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bc0076220 0x7f9bc019e9c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:58.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:58.179+0000 7f9bc7e00640 1 --2- 172.21.15.159:0/689393071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bc0074c50 0x7f9bc019e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:24:58.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:58.179+0000 7f9bc7e00640 1 -- 172.21.15.159:0/689393071 >> 172.21.15.159:0/689393071 conn(0x7f9bc00fd390 msgr2=0x7f9bc0101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:24:58.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:58.179+0000 7f9bc7e00640 1 -- 172.21.15.159:0/689393071 shutdown_connections 2024-09-19T07:24:58.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:24:58.179+0000 7f9bc7e00640 1 -- 172.21.15.159:0/689393071 wait complete. 2024-09-19T07:24:58.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:58 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:58.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:58 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:58.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:58 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:58.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:58 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:58.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:58 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:58.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:58 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:58.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:58 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:24:58.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:58 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:58.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:58 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:24:58.990 DEBUG:teuthology.orchestra.run.smithi159:osd.0> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T07:24:58.995 INFO:tasks.cephadm:Deploying osd.1 on smithi189 with /dev/vg_nvme/lv_4... 2024-09-19T07:24:58.995 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T07:24:59.245 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:24:59.261 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:24:59 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-0[44311]: 2024-09-19T07:24:59.125+0000 7f47eb25a640 -1 osd.0 0 log_to_monitors true 2024-09-19T07:24:59.556 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:59 smithi159 ceph-mon[30472]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:59.556 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:59 smithi159 ceph-mon[30472]: from='osd.0 [v2:172.21.15.159:6802/2738900830,v1:172.21.15.159:6803/2738900830]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T07:24:59.556 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:24:59 smithi159 ceph-mon[30472]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T07:24:59.680 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:59 smithi189 ceph-mon[34851]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:59.680 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:59 smithi189 ceph-mon[34851]: from='osd.0 [v2:172.21.15.159:6802/2738900830,v1:172.21.15.159:6803/2738900830]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T07:24:59.680 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:24:59 smithi189 ceph-mon[34851]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T07:24:59.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:59 smithi203 ceph-mon[35335]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:24:59.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:59 smithi203 ceph-mon[35335]: from='osd.0 [v2:172.21.15.159:6802/2738900830,v1:172.21.15.159:6803/2738900830]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T07:24:59.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:24:59 smithi203 ceph-mon[35335]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T07:25:00.553 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:00 smithi189 ceph-mon[34851]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T07:25:00.553 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:00 smithi189 ceph-mon[34851]: from='osd.0 [v2:172.21.15.159:6802/2738900830,v1:172.21.15.159:6803/2738900830]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-19T07:25:00.553 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:00 smithi189 ceph-mon[34851]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T07:25:00.553 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:00 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:25:00.554 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:00 smithi189 ceph-mon[34851]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-19T07:25:00.667 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:00 smithi159 ceph-mon[30472]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T07:25:00.667 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:00 smithi159 ceph-mon[30472]: from='osd.0 [v2:172.21.15.159:6802/2738900830,v1:172.21.15.159:6803/2738900830]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-19T07:25:00.667 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:00 smithi159 ceph-mon[30472]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T07:25:00.667 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:00 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:25:00.668 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:00 smithi159 ceph-mon[30472]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-19T07:25:00.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:00 smithi203 ceph-mon[35335]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T07:25:00.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:00 smithi203 ceph-mon[35335]: from='osd.0 [v2:172.21.15.159:6802/2738900830,v1:172.21.15.159:6803/2738900830]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-19T07:25:00.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:00 smithi203 ceph-mon[35335]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T07:25:00.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:00 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:25:00.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:00 smithi203 ceph-mon[35335]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-19T07:25:01.346 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:25:00 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-0[44311]: 2024-09-19T07:25:00.929+0000 7f47e6800640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T07:25:01.346 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:25:00 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-0[44311]: 2024-09-19T07:25:00.940+0000 7f47dbe00640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:25:01.585 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:25:01.609 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_4 2024-09-19T07:25:01.633 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:01 smithi189 ceph-mon[34851]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-19T07:25:01.633 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:01 smithi189 ceph-mon[34851]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T07:25:01.633 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:25:01.633 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:25:01.633 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:01 smithi189 ceph-mon[34851]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:25:01.633 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:01 smithi189 ceph-mon[34851]: from='osd.0 ' entity='osd.0' 2024-09-19T07:25:01.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:01 smithi159 ceph-mon[30472]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-19T07:25:01.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:01 smithi159 ceph-mon[30472]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T07:25:01.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:25:01.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:25:01.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:01 smithi159 ceph-mon[30472]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:25:01.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:01 smithi159 ceph-mon[30472]: from='osd.0 ' entity='osd.0' 2024-09-19T07:25:01.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:01 smithi203 ceph-mon[35335]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-19T07:25:01.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:01 smithi203 ceph-mon[35335]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T07:25:01.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:25:01.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:25:01.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:01 smithi203 ceph-mon[35335]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:25:01.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:01 smithi203 ceph-mon[35335]: from='osd.0 ' entity='osd.0' 2024-09-19T07:25:01.851 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:25:02.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.327+0000 7f1da6767640 1 -- 172.21.15.189:0/2077891847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1da01037c0 msgr2=0x7f1da010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:25:02.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.327+0000 7f1da6767640 1 --2- 172.21.15.189:0/2077891847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1da01037c0 0x7f1da010a050 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009ca0 tx=0x7f1d9802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:25:02.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.327+0000 7f1da6767640 1 -- 172.21.15.189:0/2077891847 shutdown_connections 2024-09-19T07:25:02.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.327+0000 7f1da6767640 1 --2- 172.21.15.189:0/2077891847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1da01037c0 0x7f1da010a050 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:02.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.327+0000 7f1da6767640 1 --2- 172.21.15.189:0/2077891847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da0102e00 0x7f1da0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:02.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.327+0000 7f1da6767640 1 --2- 172.21.15.189:0/2077891847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1da0101c00 0x7f1da0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:02.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.327+0000 7f1da6767640 1 -- 172.21.15.189:0/2077891847 >> 172.21.15.189:0/2077891847 conn(0x7f1da00fd390 msgr2=0x7f1da00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:25:02.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.327+0000 7f1da6767640 1 -- 172.21.15.189:0/2077891847 shutdown_connections 2024-09-19T07:25:02.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.327+0000 7f1da6767640 1 -- 172.21.15.189:0/2077891847 wait complete. 2024-09-19T07:25:02.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.328+0000 7f1da6767640 1 Processor -- start 2024-09-19T07:25:02.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.328+0000 7f1da6767640 1 -- start start 2024-09-19T07:25:02.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.328+0000 7f1da6767640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da0101c00 0x7f1da0197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:25:02.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.328+0000 7f1da6767640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1da0102e00 0x7f1da0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:25:02.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.328+0000 7f1da6767640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1da01037c0 0x7f1da019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:25:02.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.328+0000 7f1da6767640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1da010d900 con 0x7f1da0101c00 2024-09-19T07:25:02.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.328+0000 7f1da6767640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1da010d780 con 0x7f1da0102e00 2024-09-19T07:25:02.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.328+0000 7f1da6767640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1da010da80 con 0x7f1da01037c0 2024-09-19T07:25:02.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.328+0000 7f1d9f7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1da0102e00 0x7f1da0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:25:02.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.328+0000 7f1d9f7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1da0102e00 0x7f1da0198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:52006/0 (socket says 172.21.15.189:52006) 2024-09-19T07:25:02.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.328+0000 7f1d9f7fe640 1 -- 172.21.15.189:0/2658776190 learned_addr learned my addr 172.21.15.189:0/2658776190 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-19T07:25:02.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.329+0000 7f1d9ffff640 1 --2- 172.21.15.189:0/2658776190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da0101c00 0x7f1da0197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:25:02.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.329+0000 7f1da4d04640 1 --2- 172.21.15.189:0/2658776190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1da01037c0 0x7f1da019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:25:02.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.329+0000 7f1d9f7fe640 1 -- 172.21.15.189:0/2658776190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1da01037c0 msgr2=0x7f1da019f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:25:02.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.329+0000 7f1d9f7fe640 1 --2- 172.21.15.189:0/2658776190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1da01037c0 0x7f1da019f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:02.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.329+0000 7f1d9f7fe640 1 -- 172.21.15.189:0/2658776190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da0101c00 msgr2=0x7f1da0197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:25:02.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.329+0000 7f1d9f7fe640 1 --2- 172.21.15.189:0/2658776190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da0101c00 0x7f1da0197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:02.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.329+0000 7f1d9f7fe640 1 -- 172.21.15.189:0/2658776190 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da019fb50 con 0x7f1da0102e00 2024-09-19T07:25:02.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.329+0000 7f1d9f7fe640 1 --2- 172.21.15.189:0/2658776190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1da0102e00 0x7f1da0198460 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400c890 tx=0x7f1d9400cc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:25:02.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.329+0000 7f1d9d7fa640 1 -- 172.21.15.189:0/2658776190 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94004220 con 0x7f1da0102e00 2024-09-19T07:25:02.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.329+0000 7f1d9d7fa640 1 -- 172.21.15.189:0/2658776190 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d940043c0 con 0x7f1da0102e00 2024-09-19T07:25:02.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.329+0000 7f1da6767640 1 -- 172.21.15.189:0/2658776190 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da019fe40 con 0x7f1da0102e00 2024-09-19T07:25:02.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.329+0000 7f1d9d7fa640 1 -- 172.21.15.189:0/2658776190 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94004a00 con 0x7f1da0102e00 2024-09-19T07:25:02.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.329+0000 7f1da6767640 1 -- 172.21.15.189:0/2658776190 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da01a0370 con 0x7f1da0102e00 2024-09-19T07:25:02.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.330+0000 7f1da6767640 1 -- 172.21.15.189:0/2658776190 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da0108fb0 con 0x7f1da0102e00 2024-09-19T07:25:02.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.331+0000 7f1d9d7fa640 1 -- 172.21.15.189:0/2658776190 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f1d940079b0 con 0x7f1da0102e00 2024-09-19T07:25:02.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.332+0000 7f1d9d7fa640 1 --2- 172.21.15.189:0/2658776190 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1d880a31e0 0x7f1d880a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:25:02.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.332+0000 7f1d9d7fa640 1 -- 172.21.15.189:0/2658776190 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f1d940cdae0 con 0x7f1da0102e00 2024-09-19T07:25:02.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.332+0000 7f1d9ffff640 1 --2- 172.21.15.189:0/2658776190 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1d880a31e0 0x7f1d880a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:25:02.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.332+0000 7f1d9ffff640 1 --2- 172.21.15.189:0/2658776190 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1d880a31e0 0x7f1d880a56a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1da0102c60 tx=0x7f1d90009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:25:02.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.336+0000 7f1d9d7fa640 1 -- 172.21.15.189:0/2658776190 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1d94014030 con 0x7f1da0102e00 2024-09-19T07:25:02.515 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:02.515+0000 7f1da6767640 1 -- 172.21.15.189:0/2658776190 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1da0062630 con 0x7f1d880a31e0 2024-09-19T07:25:02.516 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:02 smithi189 ceph-mon[34851]: purged_snaps scrub starts 2024-09-19T07:25:02.516 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:02 smithi189 ceph-mon[34851]: purged_snaps scrub ok 2024-09-19T07:25:02.516 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:02 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:25:02.516 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:02 smithi189 ceph-mon[34851]: osd.0 [v2:172.21.15.159:6802/2738900830,v1:172.21.15.159:6803/2738900830] boot 2024-09-19T07:25:02.516 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:02 smithi189 ceph-mon[34851]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T07:25:02.516 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:02 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:25:02.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:02 smithi159 ceph-mon[30472]: purged_snaps scrub starts 2024-09-19T07:25:02.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:02 smithi159 ceph-mon[30472]: purged_snaps scrub ok 2024-09-19T07:25:02.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:02 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:25:02.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:02 smithi159 ceph-mon[30472]: osd.0 [v2:172.21.15.159:6802/2738900830,v1:172.21.15.159:6803/2738900830] boot 2024-09-19T07:25:02.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:02 smithi159 ceph-mon[30472]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T07:25:02.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:02 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:25:02.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:02 smithi203 ceph-mon[35335]: purged_snaps scrub starts 2024-09-19T07:25:02.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:02 smithi203 ceph-mon[35335]: purged_snaps scrub ok 2024-09-19T07:25:02.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:02 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:25:02.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:02 smithi203 ceph-mon[35335]: osd.0 [v2:172.21.15.159:6802/2738900830,v1:172.21.15.159:6803/2738900830] boot 2024-09-19T07:25:02.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:02 smithi203 ceph-mon[35335]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T07:25:02.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:02 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:25:03.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:03 smithi189 ceph-mon[34851]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:25:03.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:03 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:25:03.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:03 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:25:03.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:03 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:03.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:03 smithi189 ceph-mon[34851]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:03.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:03 smithi189 ceph-mon[34851]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T07:25:03.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:03 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:03.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:03 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:03.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:03 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:25:03.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:03 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:03.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:03 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:03.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:03 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:25:03.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:03 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:03.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:03 smithi159 ceph-mon[30472]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:25:03.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:03 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:25:03.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:03 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:25:03.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:03 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:03.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:03 smithi159 ceph-mon[30472]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:03.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:03 smithi159 ceph-mon[30472]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T07:25:03.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:03 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:03.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:03 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:03.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:03 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:25:03.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:03 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:03.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:03 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:03.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:03 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:25:03.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:03 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:03.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:03 smithi203 ceph-mon[35335]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:25:03.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:03 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:25:03.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:03 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:25:03.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:03 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:03.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:03 smithi203 ceph-mon[35335]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:03.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:03 smithi203 ceph-mon[35335]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T07:25:03.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:03 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:03.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:03 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:03.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:03 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:25:03.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:03 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:03.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:03 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:03.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:03 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:25:03.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:03 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:04.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:04 smithi203 ceph-mon[35335]: Detected new or changed devices on smithi159 2024-09-19T07:25:04.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:04 smithi203 ceph-mon[35335]: Adjusting osd_memory_target on smithi159 to 17089M 2024-09-19T07:25:04.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:04 smithi189 ceph-mon[34851]: Detected new or changed devices on smithi159 2024-09-19T07:25:04.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:04 smithi189 ceph-mon[34851]: Adjusting osd_memory_target on smithi159 to 17089M 2024-09-19T07:25:04.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:04 smithi159 ceph-mon[30472]: Detected new or changed devices on smithi159 2024-09-19T07:25:04.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:04 smithi159 ceph-mon[30472]: Adjusting osd_memory_target on smithi159 to 17089M 2024-09-19T07:25:05.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:05 smithi203 ceph-mon[35335]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:05.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:05 smithi203 ceph-mon[35335]: from='client.? 172.21.15.189:0/482570590' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7513adda-ca82-4df4-aa99-3b050b410424"}]: dispatch 2024-09-19T07:25:05.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:05 smithi203 ceph-mon[35335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7513adda-ca82-4df4-aa99-3b050b410424"}]: dispatch 2024-09-19T07:25:05.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:05 smithi203 ceph-mon[35335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7513adda-ca82-4df4-aa99-3b050b410424"}]': finished 2024-09-19T07:25:05.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:05 smithi203 ceph-mon[35335]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T07:25:05.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:05 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:25:05.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:05 smithi203 ceph-mon[35335]: from='client.? 172.21.15.189:0/207717565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:25:05.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:05 smithi189 ceph-mon[34851]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:05.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:05 smithi189 ceph-mon[34851]: from='client.? 172.21.15.189:0/482570590' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7513adda-ca82-4df4-aa99-3b050b410424"}]: dispatch 2024-09-19T07:25:05.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:05 smithi189 ceph-mon[34851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7513adda-ca82-4df4-aa99-3b050b410424"}]: dispatch 2024-09-19T07:25:05.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:05 smithi189 ceph-mon[34851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7513adda-ca82-4df4-aa99-3b050b410424"}]': finished 2024-09-19T07:25:05.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:05 smithi189 ceph-mon[34851]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T07:25:05.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:05 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:25:05.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:05 smithi189 ceph-mon[34851]: from='client.? 172.21.15.189:0/207717565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:25:05.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:05 smithi159 ceph-mon[30472]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:05.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:05 smithi159 ceph-mon[30472]: from='client.? 172.21.15.189:0/482570590' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7513adda-ca82-4df4-aa99-3b050b410424"}]: dispatch 2024-09-19T07:25:05.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:05 smithi159 ceph-mon[30472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7513adda-ca82-4df4-aa99-3b050b410424"}]: dispatch 2024-09-19T07:25:05.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:05 smithi159 ceph-mon[30472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7513adda-ca82-4df4-aa99-3b050b410424"}]': finished 2024-09-19T07:25:05.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:05 smithi159 ceph-mon[30472]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T07:25:05.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:05 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:25:05.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:05 smithi159 ceph-mon[30472]: from='client.? 172.21.15.189:0/207717565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:25:08.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:07 smithi203 ceph-mon[35335]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:08.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:07 smithi189 ceph-mon[34851]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:08.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:07 smithi159 ceph-mon[30472]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:09.792 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:09 smithi189 ceph-mon[34851]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:10.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:09 smithi203 ceph-mon[35335]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:10.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:09 smithi159 ceph-mon[30472]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:11.707 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:11 smithi189 ceph-mon[34851]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:12.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:11 smithi203 ceph-mon[35335]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:12.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:11 smithi159 ceph-mon[30472]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:13.005 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T07:25:13.005 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:13.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T07:25:13.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:13.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T07:25:13.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:14.008 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:13 smithi189 ceph-mon[34851]: Deploying daemon osd.1 on smithi189 2024-09-19T07:25:14.008 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:13 smithi189 ceph-mon[34851]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:14.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:13 smithi203 ceph-mon[35335]: Deploying daemon osd.1 on smithi189 2024-09-19T07:25:14.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:13 smithi203 ceph-mon[35335]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:14.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:13 smithi159 ceph-mon[30472]: Deploying daemon osd.1 on smithi189 2024-09-19T07:25:14.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:13 smithi159 ceph-mon[30472]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:15.997 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:15 smithi189 ceph-mon[34851]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:16.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:15 smithi203 ceph-mon[35335]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:16.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:15 smithi159 ceph-mon[30472]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:18.025 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:17 smithi189 ceph-mon[34851]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:18.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:17 smithi203 ceph-mon[35335]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:18.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:17 smithi159 ceph-mon[30472]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:20.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:19 smithi203 ceph-mon[35335]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:20.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:19 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:25:20.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:19 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:20.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:19 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:20.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:19 smithi189 ceph-mon[34851]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:20.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:19 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:25:20.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:19 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:20.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:19 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:20.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:19 smithi159 ceph-mon[30472]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:20.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:19 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:25:20.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:19 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:20.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:19 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:21 smithi203 ceph-mon[35335]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:25:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:22.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:21 smithi189 ceph-mon[34851]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:22.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:22.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:22.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:22.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:25:22.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:22.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:21 smithi159 ceph-mon[30472]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:22.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:22.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:22.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:22.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:25:22.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:23.296 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:23.296+0000 7f1d9d7fa640 1 -- 172.21.15.189:0/2658776190 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1da0062630 con 0x7f1d880a31e0 2024-09-19T07:25:23.296 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 1 on host 'smithi189' 2024-09-19T07:25:23.299 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:23.299+0000 7f1da6767640 1 -- 172.21.15.189:0/2658776190 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1d880a31e0 msgr2=0x7f1d880a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:25:23.299 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:23.299+0000 7f1da6767640 1 --2- 172.21.15.189:0/2658776190 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1d880a31e0 0x7f1d880a56a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1da0102c60 tx=0x7f1d90009110 comp rx=0 tx=0).stop 2024-09-19T07:25:23.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:23.299+0000 7f1da6767640 1 -- 172.21.15.189:0/2658776190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1da0102e00 msgr2=0x7f1da0198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:25:23.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:23.299+0000 7f1da6767640 1 --2- 172.21.15.189:0/2658776190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1da0102e00 0x7f1da0198460 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400c890 tx=0x7f1d9400cc90 comp rx=0 tx=0).stop 2024-09-19T07:25:23.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:23.299+0000 7f1da6767640 1 -- 172.21.15.189:0/2658776190 shutdown_connections 2024-09-19T07:25:23.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:23.299+0000 7f1da6767640 1 --2- 172.21.15.189:0/2658776190 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1d880a31e0 0x7f1d880a56a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:23.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:23.299+0000 7f1da6767640 1 --2- 172.21.15.189:0/2658776190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1da01037c0 0x7f1da019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:23.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:23.299+0000 7f1da6767640 1 --2- 172.21.15.189:0/2658776190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1da0102e00 0x7f1da0198460 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:23.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:23.299+0000 7f1da6767640 1 --2- 172.21.15.189:0/2658776190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da0101c00 0x7f1da0197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:23.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:23.299+0000 7f1da6767640 1 -- 172.21.15.189:0/2658776190 >> 172.21.15.189:0/2658776190 conn(0x7f1da00fd390 msgr2=0x7f1da0106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:25:23.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:23.299+0000 7f1da6767640 1 -- 172.21.15.189:0/2658776190 shutdown_connections 2024-09-19T07:25:23.300 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:25:23.299+0000 7f1da6767640 1 -- 172.21.15.189:0/2658776190 wait complete. 2024-09-19T07:25:23.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:23 smithi159 ceph-mon[30472]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:23.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:23 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:25:23.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:23 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:23.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:23 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:23.921 DEBUG:teuthology.orchestra.run.smithi189:osd.1> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T07:25:23.964 INFO:tasks.cephadm:Deploying osd.2 on smithi203 with /dev/vg_nvme/lv_4... 2024-09-19T07:25:23.964 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T07:25:23.984 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:23 smithi189 ceph-mon[34851]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:23.985 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:23 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:25:23.985 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:23 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:23.985 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:23 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:23.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:23 smithi203 ceph-mon[35335]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:23.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:23 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:25:23.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:23 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:23.993 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:23 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:24.218 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:25:24.556 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:25:24 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-1[39332]: 2024-09-19T07:25:24.452+0000 7f511dec8640 -1 osd.1 0 log_to_monitors true 2024-09-19T07:25:25.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:24 smithi203 ceph-mon[35335]: from='osd.1 [v2:172.21.15.189:6800/265696178,v1:172.21.15.189:6801/265696178]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T07:25:25.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:24 smithi203 ceph-mon[35335]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T07:25:25.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:24 smithi189 ceph-mon[34851]: from='osd.1 [v2:172.21.15.189:6800/265696178,v1:172.21.15.189:6801/265696178]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T07:25:25.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:24 smithi189 ceph-mon[34851]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T07:25:25.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:24 smithi159 ceph-mon[30472]: from='osd.1 [v2:172.21.15.189:6800/265696178,v1:172.21.15.189:6801/265696178]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T07:25:25.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:24 smithi159 ceph-mon[30472]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T07:25:25.892 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:25 smithi203 ceph-mon[35335]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:25.892 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:25 smithi203 ceph-mon[35335]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T07:25:25.892 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:25 smithi203 ceph-mon[35335]: from='osd.1 [v2:172.21.15.189:6800/265696178,v1:172.21.15.189:6801/265696178]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:25:25.892 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:25 smithi203 ceph-mon[35335]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T07:25:25.892 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:25 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:25:25.893 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:25 smithi203 ceph-mon[35335]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:25:26.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:25 smithi189 ceph-mon[34851]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:26.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:25 smithi189 ceph-mon[34851]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T07:25:26.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:25 smithi189 ceph-mon[34851]: from='osd.1 [v2:172.21.15.189:6800/265696178,v1:172.21.15.189:6801/265696178]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:25:26.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:25 smithi189 ceph-mon[34851]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T07:25:26.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:25 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:25:26.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:25 smithi189 ceph-mon[34851]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:25:26.057 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:25:25 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-1[39332]: 2024-09-19T07:25:25.713+0000 7f5119e4b640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T07:25:26.057 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:25:25 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-1[39332]: 2024-09-19T07:25:25.717+0000 7f5115c73640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:25:26.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:25 smithi159 ceph-mon[30472]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:26.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:25 smithi159 ceph-mon[30472]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T07:25:26.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:25 smithi159 ceph-mon[30472]: from='osd.1 [v2:172.21.15.189:6800/265696178,v1:172.21.15.189:6801/265696178]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:25:26.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:25 smithi159 ceph-mon[30472]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T07:25:26.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:25 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:25:26.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:25 smithi159 ceph-mon[30472]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:25:26.673 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:25:26.700 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_4 2024-09-19T07:25:26.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:26 smithi189 ceph-mon[34851]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-19T07:25:26.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:26 smithi189 ceph-mon[34851]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T07:25:26.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:26 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:25:26.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:26 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:25:26.985 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:25:27.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:26 smithi203 ceph-mon[35335]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-19T07:25:27.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:26 smithi203 ceph-mon[35335]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T07:25:27.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:26 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:25:27.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:26 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:25:27.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:26 smithi159 ceph-mon[30472]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-19T07:25:27.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:26 smithi159 ceph-mon[30472]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T07:25:27.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:26 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:25:27.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:26 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:25:27.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.514+0000 7fa854aaf640 1 -- 172.21.15.203:0/1151094005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8501020a0 msgr2=0x7fa8501024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:25:27.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.514+0000 7fa854aaf640 1 --2- 172.21.15.203:0/1151094005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8501020a0 0x7fa8501024a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa840009900 tx=0x7fa84002a410 comp rx=0 tx=0).stop 2024-09-19T07:25:27.524 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.514+0000 7fa854aaf640 1 -- 172.21.15.203:0/1151094005 shutdown_connections 2024-09-19T07:25:27.524 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.514+0000 7fa854aaf640 1 --2- 172.21.15.203:0/1151094005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8501017a0 0x7fa850109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:27.524 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.514+0000 7fa854aaf640 1 --2- 172.21.15.203:0/1151094005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa850100de0 0x7fa850101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:27.524 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.514+0000 7fa854aaf640 1 --2- 172.21.15.203:0/1151094005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8501020a0 0x7fa8501024a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:27.524 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.514+0000 7fa854aaf640 1 -- 172.21.15.203:0/1151094005 >> 172.21.15.203:0/1151094005 conn(0x7fa8500fba40 msgr2=0x7fa8500fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:25:27.524 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.514+0000 7fa854aaf640 1 -- 172.21.15.203:0/1151094005 shutdown_connections 2024-09-19T07:25:27.524 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.514+0000 7fa854aaf640 1 -- 172.21.15.203:0/1151094005 wait complete. 2024-09-19T07:25:27.524 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.515+0000 7fa854aaf640 1 Processor -- start 2024-09-19T07:25:27.524 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.515+0000 7fa854aaf640 1 -- start start 2024-09-19T07:25:27.524 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.515+0000 7fa854aaf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa850100de0 0x7fa85010b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:25:27.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.515+0000 7fa854aaf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8501017a0 0x7fa850109ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:25:27.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.515+0000 7fa854aaf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8501020a0 0x7fa85010a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:25:27.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.515+0000 7fa854aaf640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa85010d910 con 0x7fa8501020a0 2024-09-19T07:25:27.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.515+0000 7fa854aaf640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa85010d790 con 0x7fa8501017a0 2024-09-19T07:25:27.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.515+0000 7fa854aaf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa85010da90 con 0x7fa850100de0 2024-09-19T07:25:27.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.515+0000 7fa84e59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa850100de0 0x7fa85010b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:25:27.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.515+0000 7fa84e59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa850100de0 0x7fa85010b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:34514/0 (socket says 172.21.15.203:34514) 2024-09-19T07:25:27.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.515+0000 7fa84e59c640 1 -- 172.21.15.203:0/576855404 learned_addr learned my addr 172.21.15.203:0/576855404 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-19T07:25:27.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.515+0000 7fa84ed9d640 1 --2- 172.21.15.203:0/576855404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8501020a0 0x7fa85010a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:25:27.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.516+0000 7fa84e59c640 1 -- 172.21.15.203:0/576855404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8501017a0 msgr2=0x7fa850109ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:25:27.526 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.516+0000 7fa84dd9b640 1 --2- 172.21.15.203:0/576855404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8501017a0 0x7fa850109ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:25:27.526 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.516+0000 7fa84e59c640 1 --2- 172.21.15.203:0/576855404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8501017a0 0x7fa850109ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:27.526 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.516+0000 7fa84e59c640 1 -- 172.21.15.203:0/576855404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8501020a0 msgr2=0x7fa85010a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:25:27.526 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.516+0000 7fa84e59c640 1 --2- 172.21.15.203:0/576855404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8501020a0 0x7fa85010a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:27.526 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.516+0000 7fa84e59c640 1 -- 172.21.15.203:0/576855404 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85010a8c0 con 0x7fa850100de0 2024-09-19T07:25:27.526 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.516+0000 7fa84ed9d640 1 --2- 172.21.15.203:0/576855404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8501020a0 0x7fa85010a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:25:27.526 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.516+0000 7fa84e59c640 1 --2- 172.21.15.203:0/576855404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa850100de0 0x7fa85010b470 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa840005ab0 tx=0x7fa840004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:25:27.526 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.516+0000 7fa83f7fe640 1 -- 172.21.15.203:0/576855404 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84003f030 con 0x7fa850100de0 2024-09-19T07:25:27.526 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.516+0000 7fa83f7fe640 1 -- 172.21.15.203:0/576855404 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa840003ce0 con 0x7fa850100de0 2024-09-19T07:25:27.526 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.516+0000 7fa854aaf640 1 -- 172.21.15.203:0/576855404 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8501a20d0 con 0x7fa850100de0 2024-09-19T07:25:27.526 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.516+0000 7fa83f7fe640 1 -- 172.21.15.203:0/576855404 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840035070 con 0x7fa850100de0 2024-09-19T07:25:27.526 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.516+0000 7fa854aaf640 1 -- 172.21.15.203:0/576855404 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8501a2650 con 0x7fa850100de0 2024-09-19T07:25:27.526 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.517+0000 7fa854aaf640 1 -- 172.21.15.203:0/576855404 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa850107fe0 con 0x7fa850100de0 2024-09-19T07:25:27.527 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.518+0000 7fa83f7fe640 1 -- 172.21.15.203:0/576855404 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7fa840042030 con 0x7fa850100de0 2024-09-19T07:25:27.527 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.519+0000 7fa83f7fe640 1 --2- 172.21.15.203:0/576855404 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa8380a3230 0x7fa8380a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:25:27.527 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.519+0000 7fa83f7fe640 1 -- 172.21.15.203:0/576855404 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7fa8400ee950 con 0x7fa850100de0 2024-09-19T07:25:27.527 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.520+0000 7fa84dd9b640 1 --2- 172.21.15.203:0/576855404 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa8380a3230 0x7fa8380a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:25:27.527 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.520+0000 7fa84dd9b640 1 --2- 172.21.15.203:0/576855404 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa8380a3230 0x7fa8380a56f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa85010b1f0 tx=0x7fa844008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:25:27.527 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.523+0000 7fa83f7fe640 1 -- 172.21.15.203:0/576855404 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa8400bcec0 con 0x7fa850100de0 2024-09-19T07:25:27.698 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:27.697+0000 7fa854aaf640 1 -- 172.21.15.203:0/576855404 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa850062630 con 0x7fa8380a3230 2024-09-19T07:25:28.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:27 smithi203 ceph-mon[35335]: purged_snaps scrub starts 2024-09-19T07:25:28.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:27 smithi203 ceph-mon[35335]: purged_snaps scrub ok 2024-09-19T07:25:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:27 smithi203 ceph-mon[35335]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:27 smithi203 ceph-mon[35335]: osd.1 [v2:172.21.15.189:6800/265696178,v1:172.21.15.189:6801/265696178] boot 2024-09-19T07:25:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:27 smithi203 ceph-mon[35335]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T07:25:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:25:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:27 smithi203 ceph-mon[35335]: Detected new or changed devices on smithi189 2024-09-19T07:25:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:25:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:25:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:28.038 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:25:28.038 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:25:28.038 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:28.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:27 smithi189 ceph-mon[34851]: purged_snaps scrub starts 2024-09-19T07:25:28.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:27 smithi189 ceph-mon[34851]: purged_snaps scrub ok 2024-09-19T07:25:28.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:27 smithi189 ceph-mon[34851]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:28.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:27 smithi189 ceph-mon[34851]: osd.1 [v2:172.21.15.189:6800/265696178,v1:172.21.15.189:6801/265696178] boot 2024-09-19T07:25:28.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:27 smithi189 ceph-mon[34851]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T07:25:28.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:25:28.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:27 smithi189 ceph-mon[34851]: Detected new or changed devices on smithi189 2024-09-19T07:25:28.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:28.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:28.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:25:28.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:28.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:28.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:25:28.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:28.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:25:28.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:25:28.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:27 smithi159 ceph-mon[30472]: purged_snaps scrub starts 2024-09-19T07:25:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:27 smithi159 ceph-mon[30472]: purged_snaps scrub ok 2024-09-19T07:25:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:27 smithi159 ceph-mon[30472]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:25:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:27 smithi159 ceph-mon[30472]: osd.1 [v2:172.21.15.189:6800/265696178,v1:172.21.15.189:6801/265696178] boot 2024-09-19T07:25:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:27 smithi159 ceph-mon[30472]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T07:25:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:25:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:27 smithi159 ceph-mon[30472]: Detected new or changed devices on smithi189 2024-09-19T07:25:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:28.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:25:28.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:28.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:28.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:25:28.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:28.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:25:28.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:25:28.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:29.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:28 smithi203 ceph-mon[35335]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-19T07:25:29.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:28 smithi203 ceph-mon[35335]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:25:29.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:28 smithi203 ceph-mon[35335]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T07:25:29.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:28 smithi189 ceph-mon[34851]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-19T07:25:29.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:28 smithi189 ceph-mon[34851]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:25:29.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:28 smithi189 ceph-mon[34851]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T07:25:29.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:28 smithi159 ceph-mon[30472]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-19T07:25:29.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:28 smithi159 ceph-mon[30472]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:25:29.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:28 smithi159 ceph-mon[30472]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T07:25:30.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:29 smithi203 ceph-mon[35335]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:30.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:29 smithi189 ceph-mon[34851]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:30.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:29 smithi159 ceph-mon[30472]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:31.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:30 smithi203 ceph-mon[35335]: from='client.? 172.21.15.203:0/82965802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa03be79-58d8-4eb7-a051-0b46c502b12d"}]: dispatch 2024-09-19T07:25:31.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:30 smithi203 ceph-mon[35335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa03be79-58d8-4eb7-a051-0b46c502b12d"}]: dispatch 2024-09-19T07:25:31.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:30 smithi203 ceph-mon[35335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa03be79-58d8-4eb7-a051-0b46c502b12d"}]': finished 2024-09-19T07:25:31.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:30 smithi203 ceph-mon[35335]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T07:25:31.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:31.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:30 smithi203 ceph-mon[35335]: from='client.? 172.21.15.203:0/2332725366' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:25:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:30 smithi189 ceph-mon[34851]: from='client.? 172.21.15.203:0/82965802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa03be79-58d8-4eb7-a051-0b46c502b12d"}]: dispatch 2024-09-19T07:25:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:30 smithi189 ceph-mon[34851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa03be79-58d8-4eb7-a051-0b46c502b12d"}]: dispatch 2024-09-19T07:25:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:30 smithi189 ceph-mon[34851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa03be79-58d8-4eb7-a051-0b46c502b12d"}]': finished 2024-09-19T07:25:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:30 smithi189 ceph-mon[34851]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T07:25:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:30 smithi189 ceph-mon[34851]: from='client.? 172.21.15.203:0/2332725366' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:25:31.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:30 smithi159 ceph-mon[30472]: from='client.? 172.21.15.203:0/82965802' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa03be79-58d8-4eb7-a051-0b46c502b12d"}]: dispatch 2024-09-19T07:25:31.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:30 smithi159 ceph-mon[30472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa03be79-58d8-4eb7-a051-0b46c502b12d"}]: dispatch 2024-09-19T07:25:31.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:30 smithi159 ceph-mon[30472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa03be79-58d8-4eb7-a051-0b46c502b12d"}]': finished 2024-09-19T07:25:31.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:30 smithi159 ceph-mon[30472]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T07:25:31.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:31.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:30 smithi159 ceph-mon[30472]: from='client.? 172.21.15.203:0/2332725366' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:25:32.023 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:31 smithi189 ceph-mon[34851]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:32.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:31 smithi203 ceph-mon[35335]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:32.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:31 smithi159 ceph-mon[30472]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:33 smithi203 ceph-mon[35335]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:34.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:33 smithi189 ceph-mon[34851]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:34.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:33 smithi159 ceph-mon[30472]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:36.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:35 smithi203 ceph-mon[35335]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:36.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:35 smithi189 ceph-mon[34851]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:36.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:35 smithi159 ceph-mon[30472]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:38.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:37 smithi203 ceph-mon[35335]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:38.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:37 smithi189 ceph-mon[34851]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:38.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:37 smithi159 ceph-mon[30472]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:38.923 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:38 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T07:25:38.924 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:38 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:39.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:38 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T07:25:39.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:38 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:39.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:38 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T07:25:39.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:38 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:39.936 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:39 smithi203 ceph-mon[35335]: Deploying daemon osd.2 on smithi203 2024-09-19T07:25:39.937 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:39 smithi203 ceph-mon[35335]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:39 smithi189 ceph-mon[34851]: Deploying daemon osd.2 on smithi203 2024-09-19T07:25:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:39 smithi189 ceph-mon[34851]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:40.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:39 smithi159 ceph-mon[30472]: Deploying daemon osd.2 on smithi203 2024-09-19T07:25:40.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:39 smithi159 ceph-mon[30472]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:41.848 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:41 smithi203 ceph-mon[35335]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:42.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:41 smithi189 ceph-mon[34851]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:42.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:41 smithi159 ceph-mon[30472]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:44.009 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:43 smithi203 ceph-mon[35335]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:44.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:43 smithi189 ceph-mon[34851]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:44.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:43 smithi159 ceph-mon[30472]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:45.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:44 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:25:45.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:44 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:45.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:44 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:45.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:44 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:25:45.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:44 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:45.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:44 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:45.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:44 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:25:45.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:44 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:45.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:44 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:46.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:45 smithi203 ceph-mon[35335]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:46.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:45 smithi189 ceph-mon[34851]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:46.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:45 smithi159 ceph-mon[30472]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:48.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:47 smithi189 ceph-mon[34851]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:48.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:48.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:48.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:48.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:25:48.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:48.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:47 smithi159 ceph-mon[30472]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:48.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:48.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:48.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:48.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:25:48.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:48.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:47 smithi203 ceph-mon[35335]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:48.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:48.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:48.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:48.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:25:48.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:48.817 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:48.816+0000 7fa83f7fe640 1 -- 172.21.15.203:0/576855404 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa850062630 con 0x7fa8380a3230 2024-09-19T07:25:48.817 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 2 on host 'smithi203' 2024-09-19T07:25:48.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:48.819+0000 7fa854aaf640 1 -- 172.21.15.203:0/576855404 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa8380a3230 msgr2=0x7fa8380a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:25:48.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:48.819+0000 7fa854aaf640 1 --2- 172.21.15.203:0/576855404 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa8380a3230 0x7fa8380a56f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa85010b1f0 tx=0x7fa844008040 comp rx=0 tx=0).stop 2024-09-19T07:25:48.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:48.819+0000 7fa854aaf640 1 -- 172.21.15.203:0/576855404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa850100de0 msgr2=0x7fa85010b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:25:48.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:48.819+0000 7fa854aaf640 1 --2- 172.21.15.203:0/576855404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa850100de0 0x7fa85010b470 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa840005ab0 tx=0x7fa840004360 comp rx=0 tx=0).stop 2024-09-19T07:25:48.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:48.819+0000 7fa854aaf640 1 -- 172.21.15.203:0/576855404 shutdown_connections 2024-09-19T07:25:48.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:48.819+0000 7fa854aaf640 1 --2- 172.21.15.203:0/576855404 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa8380a3230 0x7fa8380a56f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:48.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:48.819+0000 7fa854aaf640 1 --2- 172.21.15.203:0/576855404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8501020a0 0x7fa85010a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:48.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:48.819+0000 7fa854aaf640 1 --2- 172.21.15.203:0/576855404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8501017a0 0x7fa850109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:48.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:48.819+0000 7fa854aaf640 1 --2- 172.21.15.203:0/576855404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa850100de0 0x7fa85010b470 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:48.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:48.819+0000 7fa854aaf640 1 -- 172.21.15.203:0/576855404 >> 172.21.15.203:0/576855404 conn(0x7fa8500fba40 msgr2=0x7fa8501051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:25:48.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:48.819+0000 7fa854aaf640 1 -- 172.21.15.203:0/576855404 shutdown_connections 2024-09-19T07:25:48.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:25:48.820+0000 7fa854aaf640 1 -- 172.21.15.203:0/576855404 wait complete. 2024-09-19T07:25:49.497 DEBUG:teuthology.orchestra.run.smithi203:osd.2> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T07:25:49.500 INFO:tasks.cephadm:Deploying osd.3 on smithi159 with /dev/vg_nvme/lv_3... 2024-09-19T07:25:49.500 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T07:25:49.870 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:25:49.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:49 smithi159 ceph-mon[30472]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:49.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:49 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:25:49.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:49 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:49.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:49 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:50.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:49 smithi203 ceph-mon[35335]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:50.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:49 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:25:50.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:49 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:50.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:49 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:50.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:49 smithi189 ceph-mon[34851]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:50.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:49 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:25:50.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:49 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:50.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:49 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:50.680 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:25:50 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-2[40318]: 2024-09-19T07:25:50.383+0000 7f9c723b1640 -1 osd.2 0 log_to_monitors true 2024-09-19T07:25:50.931 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:50 smithi203 ceph-mon[35335]: from='osd.2 [v2:172.21.15.203:6800/1826565288,v1:172.21.15.203:6801/1826565288]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T07:25:50.931 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:50 smithi203 ceph-mon[35335]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T07:25:51.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:50 smithi189 ceph-mon[34851]: from='osd.2 [v2:172.21.15.203:6800/1826565288,v1:172.21.15.203:6801/1826565288]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T07:25:51.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:50 smithi189 ceph-mon[34851]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T07:25:51.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:50 smithi159 ceph-mon[30472]: from='osd.2 [v2:172.21.15.203:6800/1826565288,v1:172.21.15.203:6801/1826565288]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T07:25:51.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:50 smithi159 ceph-mon[30472]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T07:25:51.976 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:51 smithi203 ceph-mon[35335]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:51.977 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:51 smithi203 ceph-mon[35335]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T07:25:51.977 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:51 smithi203 ceph-mon[35335]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T07:25:51.977 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:51 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:51.977 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:51 smithi203 ceph-mon[35335]: from='osd.2 [v2:172.21.15.203:6800/1826565288,v1:172.21.15.203:6801/1826565288]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:25:51.977 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:51 smithi203 ceph-mon[35335]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:25:51.977 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:25:51 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-2[40318]: 2024-09-19T07:25:51.804+0000 7f9c6e334640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T07:25:51.977 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:25:51 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-2[40318]: 2024-09-19T07:25:51.809+0000 7f9c6a15c640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:25:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:51 smithi159 ceph-mon[30472]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:51 smithi159 ceph-mon[30472]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T07:25:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:51 smithi159 ceph-mon[30472]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T07:25:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:51 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:51 smithi159 ceph-mon[30472]: from='osd.2 [v2:172.21.15.203:6800/1826565288,v1:172.21.15.203:6801/1826565288]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:25:52.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:51 smithi159 ceph-mon[30472]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:25:52.232 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:51 smithi189 ceph-mon[34851]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:52.233 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:51 smithi189 ceph-mon[34851]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T07:25:52.233 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:51 smithi189 ceph-mon[34851]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T07:25:52.233 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:51 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:52.233 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:51 smithi189 ceph-mon[34851]: from='osd.2 [v2:172.21.15.203:6800/1826565288,v1:172.21.15.203:6801/1826565288]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:25:52.233 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:51 smithi189 ceph-mon[34851]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:25:53.055 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:52 smithi159 ceph-mon[30472]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-19T07:25:53.055 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:52 smithi159 ceph-mon[30472]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T07:25:53.055 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:52 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:53.055 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:52 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:53.055 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:52 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:53.056 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:52 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:53.056 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:52 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:25:53.056 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:52 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:53.056 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:52 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:53.056 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:52 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:25:53.056 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:52 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:53.056 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:52 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:53.199 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:52 smithi189 ceph-mon[34851]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-19T07:25:53.199 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:52 smithi189 ceph-mon[34851]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T07:25:53.199 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:52 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:53.199 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:52 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:53.199 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:52 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:53.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:52 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:53.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:52 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:25:53.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:52 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:53.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:52 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:53.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:52 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:25:53.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:52 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:53.200 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:52 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:53.220 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:25:53.253 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi159:vg_nvme/lv_3 2024-09-19T07:25:53.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:52 smithi203 ceph-mon[35335]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-19T07:25:53.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:52 smithi203 ceph-mon[35335]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T07:25:53.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:52 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:53.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:52 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:53.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:52 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:53.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:52 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:53.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:52 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:25:53.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:52 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:53.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:52 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:53.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:52 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:25:53.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:52 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:53.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:52 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:53.634 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:25:54.108 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:53 smithi159 ceph-mon[30472]: purged_snaps scrub starts 2024-09-19T07:25:54.108 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:53 smithi159 ceph-mon[30472]: purged_snaps scrub ok 2024-09-19T07:25:54.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:53 smithi159 ceph-mon[30472]: Detected new or changed devices on smithi203 2024-09-19T07:25:54.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:53 smithi159 ceph-mon[30472]: Adjusting osd_memory_target on smithi203 to 17088M 2024-09-19T07:25:54.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:53 smithi159 ceph-mon[30472]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:54.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:53 smithi159 ceph-mon[30472]: osd.2 [v2:172.21.15.203:6800/1826565288,v1:172.21.15.203:6801/1826565288] boot 2024-09-19T07:25:54.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:53 smithi159 ceph-mon[30472]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T07:25:54.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:53 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:54.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.223+0000 7fe394a00640 1 -- 172.21.15.159:0/3798170463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe390101c00 msgr2=0x7fe390102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:25:54.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.223+0000 7fe394a00640 1 --2- 172.21.15.159:0/3798170463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe390101c00 0x7fe390102000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe380009900 tx=0x7fe38002a3f0 comp rx=0 tx=0).stop 2024-09-19T07:25:54.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.224+0000 7fe394a00640 1 -- 172.21.15.159:0/3798170463 shutdown_connections 2024-09-19T07:25:54.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.224+0000 7fe394a00640 1 --2- 172.21.15.159:0/3798170463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3901037c0 0x7fe39010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:54.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.224+0000 7fe394a00640 1 --2- 172.21.15.159:0/3798170463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe390102e00 0x7fe390103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:54.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.224+0000 7fe394a00640 1 --2- 172.21.15.159:0/3798170463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe390101c00 0x7fe390102000 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:54.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.224+0000 7fe394a00640 1 -- 172.21.15.159:0/3798170463 >> 172.21.15.159:0/3798170463 conn(0x7fe3900fd390 msgr2=0x7fe3900ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:25:54.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.224+0000 7fe394a00640 1 -- 172.21.15.159:0/3798170463 shutdown_connections 2024-09-19T07:25:54.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.224+0000 7fe394a00640 1 -- 172.21.15.159:0/3798170463 wait complete. 2024-09-19T07:25:54.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.225+0000 7fe394a00640 1 Processor -- start 2024-09-19T07:25:54.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.225+0000 7fe394a00640 1 -- start start 2024-09-19T07:25:54.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.225+0000 7fe394a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe390101c00 0x7fe390075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:25:54.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.225+0000 7fe394a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe390102e00 0x7fe3900745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:25:54.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.225+0000 7fe394a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3901037c0 0x7fe390074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:25:54.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.225+0000 7fe394a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe39010d8d0 con 0x7fe3901037c0 2024-09-19T07:25:54.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.225+0000 7fe394a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe39010d750 con 0x7fe390102e00 2024-09-19T07:25:54.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.225+0000 7fe394a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe39010da50 con 0x7fe390101c00 2024-09-19T07:25:54.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.226+0000 7fe38e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3901037c0 0x7fe390074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:25:54.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.226+0000 7fe38de00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe390101c00 0x7fe390075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:25:54.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.226+0000 7fe385400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe390102e00 0x7fe3900745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:25:54.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.226+0000 7fe38de00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe390101c00 0x7fe390075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:53286/0 (socket says 172.21.15.159:53286) 2024-09-19T07:25:54.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.226+0000 7fe38e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3901037c0 0x7fe390074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59718/0 (socket says 172.21.15.159:59718) 2024-09-19T07:25:54.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.226+0000 7fe38de00640 1 -- 172.21.15.159:0/386455923 learned_addr learned my addr 172.21.15.159:0/386455923 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:25:54.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.226+0000 7fe38de00640 1 -- 172.21.15.159:0/386455923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe390102e00 msgr2=0x7fe3900745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:25:54.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.226+0000 7fe38de00640 1 --2- 172.21.15.159:0/386455923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe390102e00 0x7fe3900745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:54.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.226+0000 7fe38de00640 1 -- 172.21.15.159:0/386455923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3901037c0 msgr2=0x7fe390074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:25:54.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.226+0000 7fe38de00640 1 --2- 172.21.15.159:0/386455923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3901037c0 0x7fe390074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:25:54.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.226+0000 7fe38de00640 1 -- 172.21.15.159:0/386455923 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3900753e0 con 0x7fe390101c00 2024-09-19T07:25:54.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.226+0000 7fe385400640 1 --2- 172.21.15.159:0/386455923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe390102e00 0x7fe3900745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:25:54.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.226+0000 7fe38de00640 1 --2- 172.21.15.159:0/386455923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe390101c00 0x7fe390075f90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe380031440 tx=0x7fe380032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:25:54.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.227+0000 7fe387400640 1 -- 172.21.15.159:0/386455923 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38003f030 con 0x7fe390101c00 2024-09-19T07:25:54.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.227+0000 7fe387400640 1 -- 172.21.15.159:0/386455923 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe380003ce0 con 0x7fe390101c00 2024-09-19T07:25:54.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.227+0000 7fe394a00640 1 -- 172.21.15.159:0/386455923 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3901a2090 con 0x7fe390101c00 2024-09-19T07:25:54.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.227+0000 7fe387400640 1 -- 172.21.15.159:0/386455923 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe380035070 con 0x7fe390101c00 2024-09-19T07:25:54.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.227+0000 7fe394a00640 1 -- 172.21.15.159:0/386455923 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3901a2540 con 0x7fe390101c00 2024-09-19T07:25:54.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.228+0000 7fe394a00640 1 -- 172.21.15.159:0/386455923 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe390109040 con 0x7fe390101c00 2024-09-19T07:25:54.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.230+0000 7fe387400640 1 -- 172.21.15.159:0/386455923 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7fe380042030 con 0x7fe390101c00 2024-09-19T07:25:54.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.231+0000 7fe387400640 1 --2- 172.21.15.159:0/386455923 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe3740a31b0 0x7fe3740a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:25:54.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.231+0000 7fe387400640 1 -- 172.21.15.159:0/386455923 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7fe3800ee490 con 0x7fe390101c00 2024-09-19T07:25:54.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.231+0000 7fe385400640 1 --2- 172.21.15.159:0/386455923 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe3740a31b0 0x7fe3740a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:25:54.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.232+0000 7fe385400640 1 --2- 172.21.15.159:0/386455923 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe3740a31b0 0x7fe3740a5670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe390075d10 tx=0x7fe378005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:25:54.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.234+0000 7fe387400640 1 -- 172.21.15.159:0/386455923 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe3800bc0f0 con 0x7fe390101c00 2024-09-19T07:25:54.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:53 smithi203 ceph-mon[35335]: purged_snaps scrub starts 2024-09-19T07:25:54.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:53 smithi203 ceph-mon[35335]: purged_snaps scrub ok 2024-09-19T07:25:54.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:53 smithi203 ceph-mon[35335]: Detected new or changed devices on smithi203 2024-09-19T07:25:54.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:53 smithi203 ceph-mon[35335]: Adjusting osd_memory_target on smithi203 to 17088M 2024-09-19T07:25:54.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:53 smithi203 ceph-mon[35335]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:54.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:53 smithi203 ceph-mon[35335]: osd.2 [v2:172.21.15.203:6800/1826565288,v1:172.21.15.203:6801/1826565288] boot 2024-09-19T07:25:54.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:53 smithi203 ceph-mon[35335]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T07:25:54.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:53 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:54.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:53 smithi189 ceph-mon[34851]: purged_snaps scrub starts 2024-09-19T07:25:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:53 smithi189 ceph-mon[34851]: purged_snaps scrub ok 2024-09-19T07:25:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:53 smithi189 ceph-mon[34851]: Detected new or changed devices on smithi203 2024-09-19T07:25:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:53 smithi189 ceph-mon[34851]: Adjusting osd_memory_target on smithi203 to 17088M 2024-09-19T07:25:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:53 smithi189 ceph-mon[34851]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:25:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:53 smithi189 ceph-mon[34851]: osd.2 [v2:172.21.15.203:6800/1826565288,v1:172.21.15.203:6801/1826565288] boot 2024-09-19T07:25:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:53 smithi189 ceph-mon[34851]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T07:25:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:53 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:25:54.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:25:54.419+0000 7fe394a00640 1 -- 172.21.15.159:0/386455923 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe390062630 con 0x7fe3740a31b0 2024-09-19T07:25:55.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:55 smithi203 ceph-mon[35335]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T07:25:55.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:55 smithi203 ceph-mon[35335]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:25:55.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:55 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:25:55.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:55 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:25:55.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:55 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:55.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:55 smithi203 ceph-mon[35335]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:25:55.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:55 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:25:55.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:55 smithi189 ceph-mon[34851]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T07:25:55.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:55 smithi189 ceph-mon[34851]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:25:55.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:55 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:25:55.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:55 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:25:55.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:55 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:55.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:55 smithi189 ceph-mon[34851]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:25:55.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:55 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:25:55.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:55 smithi159 ceph-mon[30472]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T07:25:55.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:55 smithi159 ceph-mon[30472]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:25:55.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:55 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:25:55.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:55 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:25:55.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:55 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:25:55.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:55 smithi159 ceph-mon[30472]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:25:55.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:55 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:25:56.499 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:56 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T07:25:56.499 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:56 smithi159 ceph-mon[30472]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T07:25:56.499 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:56 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:25:56.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:56 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T07:25:56.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:56 smithi203 ceph-mon[35335]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T07:25:56.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:56 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:25:56.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:56 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T07:25:56.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:56 smithi189 ceph-mon[34851]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T07:25:56.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:56 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:25:57.613 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T07:25:57.613 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:57 smithi159 ceph-mon[30472]: osdmap e21: 3 total, 3 up, 3 in 2024-09-19T07:25:57.613 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:57 smithi159 ceph-mon[30472]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:25:57.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:57 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T07:25:57.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:57 smithi203 ceph-mon[35335]: osdmap e21: 3 total, 3 up, 3 in 2024-09-19T07:25:57.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:57 smithi203 ceph-mon[35335]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:25:57.805 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:57 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T07:25:57.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:57 smithi189 ceph-mon[34851]: osdmap e21: 3 total, 3 up, 3 in 2024-09-19T07:25:57.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:57 smithi189 ceph-mon[34851]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:25:57.887 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:25:57 smithi159 sudo[49157]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T07:25:57.887 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:25:57 smithi159 sudo[49157]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:25:57.887 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:25:57 smithi159 sudo[49157]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:25:57.887 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:25:57 smithi159 sudo[49157]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:25:58.335 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:25:58 smithi159 sudo[49191]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T07:25:58.335 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:25:58 smithi159 sudo[49191]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:25:58.336 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:25:58 smithi159 sudo[49191]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:25:58.336 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:25:58 smithi159 sudo[49191]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:25:58.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:58 smithi159 ceph-mon[30472]: osdmap e22: 3 total, 3 up, 3 in 2024-09-19T07:25:58.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:58 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/426324868' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcd734f1-f40f-4566-82ab-c4607039f23f"}]: dispatch 2024-09-19T07:25:58.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:58 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/426324868' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcd734f1-f40f-4566-82ab-c4607039f23f"}]': finished 2024-09-19T07:25:58.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:58 smithi159 ceph-mon[30472]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T07:25:58.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:58 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:25:58.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:58 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:58.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:58 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/150800590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:25:58.717 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:25:58 smithi189 sudo[43026]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T07:25:58.717 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:25:58 smithi189 sudo[43026]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:25:58.718 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:25:58 smithi189 sudo[43026]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:25:58.718 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:25:58 smithi189 sudo[43026]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:25:58.718 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:58 smithi189 ceph-mon[34851]: osdmap e22: 3 total, 3 up, 3 in 2024-09-19T07:25:58.718 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:58 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/426324868' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcd734f1-f40f-4566-82ab-c4607039f23f"}]: dispatch 2024-09-19T07:25:58.718 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:58 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/426324868' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcd734f1-f40f-4566-82ab-c4607039f23f"}]': finished 2024-09-19T07:25:58.718 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:58 smithi189 ceph-mon[34851]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T07:25:58.718 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:58 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:25:58.718 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:58 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:58.718 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:25:58 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/150800590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:25:58.724 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:58 smithi203 ceph-mon[35335]: osdmap e22: 3 total, 3 up, 3 in 2024-09-19T07:25:58.724 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:58 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/426324868' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcd734f1-f40f-4566-82ab-c4607039f23f"}]: dispatch 2024-09-19T07:25:58.724 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:58 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/426324868' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcd734f1-f40f-4566-82ab-c4607039f23f"}]': finished 2024-09-19T07:25:58.724 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:58 smithi203 ceph-mon[35335]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T07:25:58.724 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:58 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:25:58.724 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:58 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:25:58.724 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:25:58 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/150800590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:25:59.056 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:25:58 smithi189 sudo[43029]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T07:25:59.056 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:25:58 smithi189 sudo[43029]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:25:59.056 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:25:58 smithi189 sudo[43029]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:25:59.056 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:25:58 smithi189 sudo[43029]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:25:59.244 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:25:58 smithi203 sudo[44390]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T07:25:59.244 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:25:58 smithi203 sudo[44390]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:25:59.244 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:25:58 smithi203 sudo[44390]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:25:59.244 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:25:59 smithi203 sudo[44390]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:25:59.536 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:25:59 smithi203 sudo[44393]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T07:25:59.536 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:25:59 smithi203 sudo[44393]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:25:59.536 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:25:59 smithi203 sudo[44393]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:25:59.536 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:25:59 smithi203 sudo[44393]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:26:00.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:59 smithi159 sudo[49615]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T07:26:00.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:59 smithi159 sudo[49615]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:26:00.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:59 smithi159 sudo[49615]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:26:00.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:25:59 smithi159 sudo[49615]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:26:00.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:00 smithi159 sudo[49629]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T07:26:00.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:00 smithi159 sudo[49629]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:26:00.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:00 smithi159 sudo[49629]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:26:00.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:00 smithi159 sudo[49629]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:26:00.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:00 smithi159 ceph-mon[30472]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:00.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:00 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:00.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:00 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:00.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:00 smithi159 ceph-mon[30472]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:26:00.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:00 smithi189 ceph-mon[34851]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:00.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:00 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:00.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:00 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:00.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:00 smithi189 ceph-mon[34851]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:26:00.850 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:00 smithi203 sudo[44396]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T07:26:00.850 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:00 smithi203 sudo[44396]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:26:00.850 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:00 smithi203 sudo[44396]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:26:00.850 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:00 smithi203 sudo[44396]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:26:00.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:26:00.862+0000 7fe387400640 1 -- 172.21.15.159:0/386455923 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fe3800b8760 con 0x7fe390101c00 2024-09-19T07:26:01.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:00 smithi203 sudo[44399]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T07:26:01.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:00 smithi203 sudo[44399]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:26:01.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:00 smithi203 sudo[44399]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:26:01.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:00 smithi203 sudo[44399]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:26:01.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:00 smithi203 ceph-mon[35335]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:01.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:00 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:01.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:00 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:01.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:00 smithi203 ceph-mon[35335]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:26:01.349 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 sudo[43032]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T07:26:01.349 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 sudo[43032]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:26:01.349 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 sudo[43032]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:26:01.349 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 sudo[43032]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:26:01.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:01 smithi159 ceph-mon[30472]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:26:01.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:26:01.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:26:01.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:26:01.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:01 smithi159 ceph-mon[30472]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:26:01.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:01 smithi159 ceph-mon[30472]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T07:26:01.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:26:01.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:26:01.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:26:01.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:01 smithi159 ceph-mon[30472]: pgmap v88: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:01.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:01 smithi159 ceph-mon[30472]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:26:01.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:01 smithi159 ceph-mon[30472]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:26:01.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:26:01.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:26:01.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:26:01.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:01 smithi159 ceph-mon[30472]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:26:01.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:01 smithi203 ceph-mon[35335]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:26:01.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:26:01.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:26:01.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:26:01.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:01 smithi203 ceph-mon[35335]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:26:01.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:01 smithi203 ceph-mon[35335]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T07:26:01.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:26:01.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:26:01.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:26:01.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:01 smithi203 ceph-mon[35335]: pgmap v88: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:01.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:01 smithi203 ceph-mon[35335]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:26:01.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:01 smithi203 ceph-mon[35335]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:26:01.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:26:01.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:26:01.788 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:26:01.788 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:01 smithi203 ceph-mon[35335]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:26:01.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 sudo[43035]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T07:26:01.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 sudo[43035]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:26:01.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 sudo[43035]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:26:01.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 sudo[43035]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:26:01.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 ceph-mon[34851]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:26:01.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:26:01.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:26:01.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:26:01.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 ceph-mon[34851]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:26:01.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 ceph-mon[34851]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T07:26:01.807 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:26:01.807 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:26:01.807 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:26:01.807 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 ceph-mon[34851]: pgmap v88: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:01.807 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 ceph-mon[34851]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:26:01.807 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 ceph-mon[34851]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:26:01.807 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:26:01.807 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:26:01.807 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:26:01.807 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:01 smithi189 ceph-mon[34851]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:26:03.979 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:03 smithi159 ceph-mon[30472]: pgmap v89: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:04.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:03 smithi203 ceph-mon[35335]: pgmap v89: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:04.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:03 smithi189 ceph-mon[34851]: pgmap v89: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:05.818 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:05 smithi159 ceph-mon[30472]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-09-19T07:26:06.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:05 smithi203 ceph-mon[35335]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-09-19T07:26:06.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:05 smithi189 ceph-mon[34851]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-09-19T07:26:08.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:07 smithi203 ceph-mon[35335]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-09-19T07:26:08.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T07:26:08.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:08.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:07 smithi203 ceph-mon[35335]: Deploying daemon osd.3 on smithi159 2024-09-19T07:26:08.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:07 smithi189 ceph-mon[34851]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-09-19T07:26:08.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T07:26:08.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:08.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:07 smithi189 ceph-mon[34851]: Deploying daemon osd.3 on smithi159 2024-09-19T07:26:08.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:07 smithi159 ceph-mon[30472]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-09-19T07:26:08.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T07:26:08.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:08.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:07 smithi159 ceph-mon[30472]: Deploying daemon osd.3 on smithi159 2024-09-19T07:26:10.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:09 smithi203 ceph-mon[35335]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-09-19T07:26:10.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:09 smithi189 ceph-mon[34851]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-09-19T07:26:10.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:09 smithi159 ceph-mon[30472]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-09-19T07:26:11.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:11 smithi159 ceph-mon[30472]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-19T07:26:12.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:11 smithi203 ceph-mon[35335]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-19T07:26:12.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:11 smithi189 ceph-mon[34851]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-19T07:26:14.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:13 smithi203 ceph-mon[35335]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-19T07:26:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:13 smithi189 ceph-mon[34851]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-19T07:26:14.111 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:13 smithi159 ceph-mon[30472]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-19T07:26:15.826 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:15 smithi159 ceph-mon[30472]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-19T07:26:15.827 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:15 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:26:15.827 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:15 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:15.827 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:15 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:16.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:15 smithi203 ceph-mon[35335]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-19T07:26:16.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:15 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:26:16.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:15 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:16.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:15 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:15 smithi189 ceph-mon[34851]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-19T07:26:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:15 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:26:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:15 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:15 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:18.029 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:17 smithi159 ceph-mon[30472]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:18.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:17 smithi203 ceph-mon[35335]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:18.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:17 smithi189 ceph-mon[34851]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:19.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:19 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:19.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:19 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:19.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:19 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:19.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:19 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:26:19.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:19 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:19.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:19 smithi159 ceph-mon[30472]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:19.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:19 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:19.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:19 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:19.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:19 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:19.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:19 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:26:19.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:19 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:19.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:19 smithi203 ceph-mon[35335]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:19.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:19 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:19.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:19 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:19.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:19 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:19.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:19 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:26:19.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:19 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:19.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:19 smithi189 ceph-mon[34851]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:19.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:26:19.881+0000 7fe387400640 1 -- 172.21.15.159:0/386455923 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe390062630 con 0x7fe3740a31b0 2024-09-19T07:26:19.881 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 3 on host 'smithi159' 2024-09-19T07:26:19.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:26:19.884+0000 7fe394a00640 1 -- 172.21.15.159:0/386455923 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe3740a31b0 msgr2=0x7fe3740a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:26:19.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:26:19.884+0000 7fe394a00640 1 --2- 172.21.15.159:0/386455923 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe3740a31b0 0x7fe3740a5670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe390075d10 tx=0x7fe378005f90 comp rx=0 tx=0).stop 2024-09-19T07:26:19.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:26:19.884+0000 7fe394a00640 1 -- 172.21.15.159:0/386455923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe390101c00 msgr2=0x7fe390075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:26:19.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:26:19.884+0000 7fe394a00640 1 --2- 172.21.15.159:0/386455923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe390101c00 0x7fe390075f90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe380031440 tx=0x7fe380032ce0 comp rx=0 tx=0).stop 2024-09-19T07:26:19.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:26:19.884+0000 7fe394a00640 1 -- 172.21.15.159:0/386455923 shutdown_connections 2024-09-19T07:26:19.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:26:19.884+0000 7fe394a00640 1 --2- 172.21.15.159:0/386455923 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe3740a31b0 0x7fe3740a5670 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:19.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:26:19.884+0000 7fe394a00640 1 --2- 172.21.15.159:0/386455923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3901037c0 0x7fe390074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:19.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:26:19.884+0000 7fe394a00640 1 --2- 172.21.15.159:0/386455923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe390102e00 0x7fe3900745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:19.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:26:19.884+0000 7fe394a00640 1 --2- 172.21.15.159:0/386455923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe390101c00 0x7fe390075f90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:19.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:26:19.884+0000 7fe394a00640 1 -- 172.21.15.159:0/386455923 >> 172.21.15.159:0/386455923 conn(0x7fe3900fd390 msgr2=0x7fe390106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:26:19.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:26:19.884+0000 7fe394a00640 1 -- 172.21.15.159:0/386455923 shutdown_connections 2024-09-19T07:26:19.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:26:19.884+0000 7fe394a00640 1 -- 172.21.15.159:0/386455923 wait complete. 2024-09-19T07:26:20.380 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:20 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:26:20.381 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:20 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:20.381 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:20 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:20.381 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:20 smithi159 ceph-mon[30472]: from='osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T07:26:20.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:20 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:26:20.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:20 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:20.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:20 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:20.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:20 smithi203 ceph-mon[35335]: from='osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T07:26:20.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:20 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:26:20.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:20 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:20.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:20 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:20.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:20 smithi189 ceph-mon[34851]: from='osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T07:26:20.684 DEBUG:teuthology.orchestra.run.smithi159:osd.3> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T07:26:20.689 INFO:tasks.cephadm:Deploying osd.4 on smithi189 with /dev/vg_nvme/lv_3... 2024-09-19T07:26:20.689 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T07:26:20.942 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:26:21.321 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:21 smithi189 ceph-mon[34851]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:21.322 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:21 smithi189 ceph-mon[34851]: from='osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T07:26:21.322 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:21 smithi189 ceph-mon[34851]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T07:26:21.322 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:21 smithi189 ceph-mon[34851]: from='osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-19T07:26:21.322 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:26:21.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:21 smithi203 ceph-mon[35335]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:21.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:21 smithi203 ceph-mon[35335]: from='osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T07:26:21.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:21 smithi203 ceph-mon[35335]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T07:26:21.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:21 smithi203 ceph-mon[35335]: from='osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-19T07:26:21.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:26:21.539 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:21 smithi159 ceph-mon[30472]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:21.539 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:21 smithi159 ceph-mon[30472]: from='osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T07:26:21.539 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:21 smithi159 ceph-mon[30472]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T07:26:21.539 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:21 smithi159 ceph-mon[30472]: from='osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-19T07:26:21.539 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:26:22.479 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:26:22 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-3[51219]: 2024-09-19T07:26:22.427+0000 7f5916000640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T07:26:22.479 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:26:22 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-3[51219]: 2024-09-19T07:26:22.443+0000 7f590c000640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:26:23.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:22 smithi189 ceph-mon[34851]: from='osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-19T07:26:23.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:22 smithi189 ceph-mon[34851]: osdmap e25: 4 total, 3 up, 4 in 2024-09-19T07:26:23.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:22 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:26:23.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:22 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:26:23.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:22 smithi189 ceph-mon[34851]: from='osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380]' entity='osd.3' 2024-09-19T07:26:23.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:22 smithi189 ceph-mon[34851]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:23.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:22 smithi189 ceph-mon[34851]: osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380] boot 2024-09-19T07:26:23.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:22 smithi189 ceph-mon[34851]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T07:26:23.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:22 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:26:23.124 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:22 smithi159 ceph-mon[30472]: from='osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-19T07:26:23.124 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:22 smithi159 ceph-mon[30472]: osdmap e25: 4 total, 3 up, 4 in 2024-09-19T07:26:23.124 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:22 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:26:23.124 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:22 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:26:23.125 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:22 smithi159 ceph-mon[30472]: from='osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380]' entity='osd.3' 2024-09-19T07:26:23.125 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:22 smithi159 ceph-mon[30472]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:23.125 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:22 smithi159 ceph-mon[30472]: osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380] boot 2024-09-19T07:26:23.125 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:22 smithi159 ceph-mon[30472]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T07:26:23.125 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:22 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:26:23.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:22 smithi203 ceph-mon[35335]: from='osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-19T07:26:23.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:22 smithi203 ceph-mon[35335]: osdmap e25: 4 total, 3 up, 4 in 2024-09-19T07:26:23.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:22 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:26:23.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:22 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:26:23.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:22 smithi203 ceph-mon[35335]: from='osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380]' entity='osd.3' 2024-09-19T07:26:23.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:22 smithi203 ceph-mon[35335]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:26:23.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:22 smithi203 ceph-mon[35335]: osd.3 [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380] boot 2024-09-19T07:26:23.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:22 smithi203 ceph-mon[35335]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T07:26:23.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:22 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:26:23.571 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:26:23.601 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_3 2024-09-19T07:26:23.848 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:26:24.139 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:23 smithi159 ceph-mon[30472]: purged_snaps scrub starts 2024-09-19T07:26:24.139 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:23 smithi159 ceph-mon[30472]: purged_snaps scrub ok 2024-09-19T07:26:24.140 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:23 smithi159 ceph-mon[30472]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T07:26:24.143 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:23 smithi189 ceph-mon[34851]: purged_snaps scrub starts 2024-09-19T07:26:24.143 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:23 smithi189 ceph-mon[34851]: purged_snaps scrub ok 2024-09-19T07:26:24.143 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:23 smithi189 ceph-mon[34851]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T07:26:24.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:23 smithi203 ceph-mon[35335]: purged_snaps scrub starts 2024-09-19T07:26:24.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:23 smithi203 ceph-mon[35335]: purged_snaps scrub ok 2024-09-19T07:26:24.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:23 smithi203 ceph-mon[35335]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T07:26:24.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.329+0000 7fb1a5f1e640 1 -- 172.21.15.189:0/460136780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1a0102690 msgr2=0x7fb1a0108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:26:24.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.329+0000 7fb1a5f1e640 1 --2- 172.21.15.189:0/460136780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1a0102690 0x7fb1a0108f20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb198009ca0 tx=0x7fb19802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:26:24.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.329+0000 7fb1a5f1e640 1 -- 172.21.15.189:0/460136780 shutdown_connections 2024-09-19T07:26:24.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.329+0000 7fb1a5f1e640 1 --2- 172.21.15.189:0/460136780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1a0102690 0x7fb1a0108f20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:24.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.329+0000 7fb1a5f1e640 1 --2- 172.21.15.189:0/460136780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1a0101cd0 0x7fb1a0102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:24.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.329+0000 7fb1a5f1e640 1 --2- 172.21.15.189:0/460136780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1a0100ad0 0x7fb1a0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:24.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.329+0000 7fb1a5f1e640 1 -- 172.21.15.189:0/460136780 >> 172.21.15.189:0/460136780 conn(0x7fb1a00fc280 msgr2=0x7fb1a00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:26:24.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.329+0000 7fb1a5f1e640 1 -- 172.21.15.189:0/460136780 shutdown_connections 2024-09-19T07:26:24.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.329+0000 7fb1a5f1e640 1 -- 172.21.15.189:0/460136780 wait complete. 2024-09-19T07:26:24.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.329+0000 7fb1a5f1e640 1 Processor -- start 2024-09-19T07:26:24.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.329+0000 7fb1a5f1e640 1 -- start start 2024-09-19T07:26:24.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb1a5f1e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1a0100ad0 0x7fb1a0196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:26:24.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb1a5f1e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1a0101cd0 0x7fb1a0197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:26:24.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb1a5f1e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1a0102690 0x7fb1a019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:26:24.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb1a5f1e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a010c820 con 0x7fb1a0100ad0 2024-09-19T07:26:24.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb1a5f1e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a010c6a0 con 0x7fb1a0101cd0 2024-09-19T07:26:24.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb1a5f1e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a010c9a0 con 0x7fb1a0102690 2024-09-19T07:26:24.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb196ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1a0101cd0 0x7fb1a0197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:26:24.347 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb196ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1a0101cd0 0x7fb1a0197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:34382/0 (socket says 172.21.15.189:34382) 2024-09-19T07:26:24.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb196ffd640 1 -- 172.21.15.189:0/798978147 learned_addr learned my addr 172.21.15.189:0/798978147 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-19T07:26:24.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb19ffff640 1 --2- 172.21.15.189:0/798978147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1a0102690 0x7fb1a019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:26:24.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb196ffd640 1 -- 172.21.15.189:0/798978147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1a0102690 msgr2=0x7fb1a019e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:26:24.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb196ffd640 1 --2- 172.21.15.189:0/798978147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1a0102690 0x7fb1a019e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:24.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb19f7fe640 1 --2- 172.21.15.189:0/798978147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1a0100ad0 0x7fb1a0196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:26:24.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb196ffd640 1 -- 172.21.15.189:0/798978147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1a0100ad0 msgr2=0x7fb1a0196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:26:24.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb196ffd640 1 --2- 172.21.15.189:0/798978147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1a0100ad0 0x7fb1a0196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:24.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb196ffd640 1 -- 172.21.15.189:0/798978147 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a019ea20 con 0x7fb1a0101cd0 2024-09-19T07:26:24.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb19f7fe640 1 --2- 172.21.15.189:0/798978147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1a0100ad0 0x7fb1a0196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:26:24.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.330+0000 7fb196ffd640 1 --2- 172.21.15.189:0/798978147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1a0101cd0 0x7fb1a0197330 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb18800c890 tx=0x7fb18800cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:26:24.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.331+0000 7fb19d7fa640 1 -- 172.21.15.189:0/798978147 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188004220 con 0x7fb1a0101cd0 2024-09-19T07:26:24.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.331+0000 7fb19d7fa640 1 -- 172.21.15.189:0/798978147 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1880043c0 con 0x7fb1a0101cd0 2024-09-19T07:26:24.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.331+0000 7fb1a5f1e640 1 -- 172.21.15.189:0/798978147 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a019ed10 con 0x7fb1a0101cd0 2024-09-19T07:26:24.349 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.331+0000 7fb19d7fa640 1 -- 172.21.15.189:0/798978147 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1880049e0 con 0x7fb1a0101cd0 2024-09-19T07:26:24.349 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.331+0000 7fb1a5f1e640 1 -- 172.21.15.189:0/798978147 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a019f270 con 0x7fb1a0101cd0 2024-09-19T07:26:24.349 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.332+0000 7fb1a5f1e640 1 -- 172.21.15.189:0/798978147 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a0107e80 con 0x7fb1a0101cd0 2024-09-19T07:26:24.349 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.333+0000 7fb19d7fa640 1 -- 172.21.15.189:0/798978147 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb1880079b0 con 0x7fb1a0101cd0 2024-09-19T07:26:24.349 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.334+0000 7fb19d7fa640 1 --2- 172.21.15.189:0/798978147 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb18c0a3270 0x7fb18c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:26:24.349 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.334+0000 7fb19d7fa640 1 -- 172.21.15.189:0/798978147 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7fb1880ce810 con 0x7fb1a0101cd0 2024-09-19T07:26:24.349 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.334+0000 7fb19f7fe640 1 --2- 172.21.15.189:0/798978147 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb18c0a3270 0x7fb18c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:26:24.349 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.335+0000 7fb19f7fe640 1 --2- 172.21.15.189:0/798978147 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb18c0a3270 0x7fb18c0a5730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb190006fd0 tx=0x7fb190008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:26:24.349 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.338+0000 7fb19d7fa640 1 -- 172.21.15.189:0/798978147 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb188014030 con 0x7fb1a0101cd0 2024-09-19T07:26:24.512 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:24.510+0000 7fb1a5f1e640 1 -- 172.21.15.189:0/798978147 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb1a0062630 con 0x7fb18c0a3270 2024-09-19T07:26:25.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:24 smithi159 ceph-mon[30472]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:26:25.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:24 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:26:25.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:24 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:26:25.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:24 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:25.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:24 smithi159 ceph-mon[30472]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:25.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:24 smithi159 ceph-mon[30472]: osdmap e28: 4 total, 4 up, 4 in 2024-09-19T07:26:25.232 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:24 smithi189 ceph-mon[34851]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:26:25.232 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:24 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:26:25.232 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:24 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:26:25.232 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:24 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:25.232 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:24 smithi189 ceph-mon[34851]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:25.232 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:24 smithi189 ceph-mon[34851]: osdmap e28: 4 total, 4 up, 4 in 2024-09-19T07:26:25.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:24 smithi203 ceph-mon[35335]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:26:25.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:24 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:26:25.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:24 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:26:25.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:24 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:25.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:24 smithi203 ceph-mon[35335]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:25.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:24 smithi203 ceph-mon[35335]: osdmap e28: 4 total, 4 up, 4 in 2024-09-19T07:26:26.157 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:25 smithi189 ceph-mon[34851]: Detected new or changed devices on smithi159 2024-09-19T07:26:26.157 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:25 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:26.158 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:25 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:26.158 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:25 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:26:26.158 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:25 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:26:26.158 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:25 smithi189 ceph-mon[34851]: Adjusting osd_memory_target on smithi159 to 8544M 2024-09-19T07:26:26.158 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:25 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:26.158 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:25 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:26.158 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:25 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:26:26.158 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:25 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:26.158 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:25 smithi189 ceph-mon[34851]: osdmap e29: 4 total, 4 up, 4 in 2024-09-19T07:26:26.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:25 smithi203 ceph-mon[35335]: Detected new or changed devices on smithi159 2024-09-19T07:26:26.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:25 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:26.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:25 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:26.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:25 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:26:26.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:25 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:26:26.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:25 smithi203 ceph-mon[35335]: Adjusting osd_memory_target on smithi159 to 8544M 2024-09-19T07:26:26.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:25 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:26.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:25 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:26.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:25 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:26:26.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:25 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:26.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:25 smithi203 ceph-mon[35335]: osdmap e29: 4 total, 4 up, 4 in 2024-09-19T07:26:26.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:25 smithi159 ceph-mon[30472]: Detected new or changed devices on smithi159 2024-09-19T07:26:26.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:25 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:26.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:25 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:26.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:25 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:26:26.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:25 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:26:26.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:25 smithi159 ceph-mon[30472]: Adjusting osd_memory_target on smithi159 to 8544M 2024-09-19T07:26:26.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:25 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:26.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:25 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:26.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:25 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:26:26.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:25 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:26.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:25 smithi159 ceph-mon[30472]: osdmap e29: 4 total, 4 up, 4 in 2024-09-19T07:26:27.805 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:27 smithi189 ceph-mon[34851]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:27.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:27 smithi189 ceph-mon[34851]: osdmap e30: 4 total, 4 up, 4 in 2024-09-19T07:26:27.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:27 smithi189 ceph-mon[34851]: from='client.? 172.21.15.189:0/3331599566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc344c9e-2267-4058-bbe1-9c270327db8a"}]: dispatch 2024-09-19T07:26:27.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:27 smithi189 ceph-mon[34851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc344c9e-2267-4058-bbe1-9c270327db8a"}]: dispatch 2024-09-19T07:26:27.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:27 smithi189 ceph-mon[34851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc344c9e-2267-4058-bbe1-9c270327db8a"}]': finished 2024-09-19T07:26:27.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:27 smithi189 ceph-mon[34851]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T07:26:27.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:27.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:27 smithi189 ceph-mon[34851]: from='client.? 172.21.15.189:0/2489599176' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:26:28.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:27 smithi203 ceph-mon[35335]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:28.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:27 smithi203 ceph-mon[35335]: osdmap e30: 4 total, 4 up, 4 in 2024-09-19T07:26:28.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:27 smithi203 ceph-mon[35335]: from='client.? 172.21.15.189:0/3331599566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc344c9e-2267-4058-bbe1-9c270327db8a"}]: dispatch 2024-09-19T07:26:28.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:27 smithi203 ceph-mon[35335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc344c9e-2267-4058-bbe1-9c270327db8a"}]: dispatch 2024-09-19T07:26:28.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:27 smithi203 ceph-mon[35335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc344c9e-2267-4058-bbe1-9c270327db8a"}]': finished 2024-09-19T07:26:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:27 smithi203 ceph-mon[35335]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T07:26:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:28.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:27 smithi203 ceph-mon[35335]: from='client.? 172.21.15.189:0/2489599176' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:26:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:27 smithi159 ceph-mon[30472]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:27 smithi159 ceph-mon[30472]: osdmap e30: 4 total, 4 up, 4 in 2024-09-19T07:26:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:27 smithi159 ceph-mon[30472]: from='client.? 172.21.15.189:0/3331599566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc344c9e-2267-4058-bbe1-9c270327db8a"}]: dispatch 2024-09-19T07:26:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:27 smithi159 ceph-mon[30472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc344c9e-2267-4058-bbe1-9c270327db8a"}]: dispatch 2024-09-19T07:26:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:27 smithi159 ceph-mon[30472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc344c9e-2267-4058-bbe1-9c270327db8a"}]': finished 2024-09-19T07:26:28.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:27 smithi159 ceph-mon[30472]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T07:26:28.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:28.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:27 smithi159 ceph-mon[30472]: from='client.? 172.21.15.189:0/2489599176' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:26:30.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:29 smithi203 ceph-mon[35335]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T07:26:30.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:29 smithi189 ceph-mon[34851]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T07:26:30.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:29 smithi159 ceph-mon[30472]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T07:26:31.785 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:31 smithi189 ceph-mon[34851]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T07:26:31.996 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:31 smithi159 ceph-mon[30472]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T07:26:32.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:31 smithi203 ceph-mon[35335]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T07:26:33.776 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:33 smithi189 ceph-mon[34851]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-19T07:26:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:33 smithi203 ceph-mon[35335]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-19T07:26:34.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:33 smithi159 ceph-mon[30472]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-19T07:26:35.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:35 smithi189 ceph-mon[34851]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T07:26:35.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T07:26:35.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:36.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:35 smithi203 ceph-mon[35335]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T07:26:36.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T07:26:36.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:36.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:35 smithi159 ceph-mon[30472]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T07:26:36.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T07:26:36.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:36.800 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:36 smithi189 ceph-mon[34851]: Deploying daemon osd.4 on smithi189 2024-09-19T07:26:37.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:36 smithi203 ceph-mon[35335]: Deploying daemon osd.4 on smithi189 2024-09-19T07:26:37.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:36 smithi159 ceph-mon[30472]: Deploying daemon osd.4 on smithi189 2024-09-19T07:26:37.738 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:37 smithi189 ceph-mon[34851]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T07:26:38.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:37 smithi203 ceph-mon[35335]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T07:26:38.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:37 smithi159 ceph-mon[30472]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T07:26:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:39 smithi203 ceph-mon[35335]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T07:26:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:39 smithi189 ceph-mon[34851]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T07:26:40.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:39 smithi159 ceph-mon[30472]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T07:26:41.932 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:41 smithi189 ceph-mon[34851]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:41.932 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:41 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:26:41.932 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:41 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:41.932 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:41 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:42.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:41 smithi203 ceph-mon[35335]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:42.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:41 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:26:42.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:41 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:42.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:41 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:42.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:41 smithi159 ceph-mon[30472]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:42.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:41 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:26:42.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:41 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:42.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:41 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:43.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:43 smithi189 ceph-mon[34851]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:44.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:43 smithi203 ceph-mon[35335]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:44.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:43 smithi159 ceph-mon[30472]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:45.509 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:45 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:45.510 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:45 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:45.510 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:45 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:45.510 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:45 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:26:45.510 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:45 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:45.510 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:45 smithi189 ceph-mon[34851]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:45.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:45 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:45.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:45 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:45.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:45 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:45.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:45 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:26:45.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:45 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:45.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:45 smithi203 ceph-mon[35335]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:45.552 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:45.552+0000 7fb19d7fa640 1 -- 172.21.15.189:0/798978147 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb1a0062630 con 0x7fb18c0a3270 2024-09-19T07:26:45.552 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 4 on host 'smithi189' 2024-09-19T07:26:45.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:45.554+0000 7fb1a5f1e640 1 -- 172.21.15.189:0/798978147 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb18c0a3270 msgr2=0x7fb18c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:26:45.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:45.554+0000 7fb1a5f1e640 1 --2- 172.21.15.189:0/798978147 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb18c0a3270 0x7fb18c0a5730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb190006fd0 tx=0x7fb190008040 comp rx=0 tx=0).stop 2024-09-19T07:26:45.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:45.554+0000 7fb1a5f1e640 1 -- 172.21.15.189:0/798978147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1a0101cd0 msgr2=0x7fb1a0197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:26:45.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:45.554+0000 7fb1a5f1e640 1 --2- 172.21.15.189:0/798978147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1a0101cd0 0x7fb1a0197330 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb18800c890 tx=0x7fb18800cd50 comp rx=0 tx=0).stop 2024-09-19T07:26:45.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:45.554+0000 7fb1a5f1e640 1 -- 172.21.15.189:0/798978147 shutdown_connections 2024-09-19T07:26:45.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:45.554+0000 7fb1a5f1e640 1 --2- 172.21.15.189:0/798978147 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb18c0a3270 0x7fb18c0a5730 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:45.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:45.554+0000 7fb1a5f1e640 1 --2- 172.21.15.189:0/798978147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1a0102690 0x7fb1a019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:45.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:45.554+0000 7fb1a5f1e640 1 --2- 172.21.15.189:0/798978147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1a0101cd0 0x7fb1a0197330 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:45.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:45.554+0000 7fb1a5f1e640 1 --2- 172.21.15.189:0/798978147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1a0100ad0 0x7fb1a0196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:45.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:45.554+0000 7fb1a5f1e640 1 -- 172.21.15.189:0/798978147 >> 172.21.15.189:0/798978147 conn(0x7fb1a00fc280 msgr2=0x7fb1a0105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:26:45.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:45.554+0000 7fb1a5f1e640 1 -- 172.21.15.189:0/798978147 shutdown_connections 2024-09-19T07:26:45.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:26:45.554+0000 7fb1a5f1e640 1 -- 172.21.15.189:0/798978147 wait complete. 2024-09-19T07:26:45.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:45 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:45.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:45 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:45.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:45 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:45.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:45 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:26:45.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:45 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:45.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:45 smithi159 ceph-mon[30472]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:46.234 DEBUG:teuthology.orchestra.run.smithi189:osd.4> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T07:26:46.236 INFO:tasks.cephadm:Deploying osd.5 on smithi203 with /dev/vg_nvme/lv_3... 2024-09-19T07:26:46.236 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T07:26:46.481 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:26:46.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:46 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:26:46.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:46 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:46.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:46 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:46.540 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:46 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:26:46.540 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:46 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:46.540 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:46 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:46.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:46 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:26:46.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:46 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:46.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:46 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:47.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:47 smithi203 ceph-mon[35335]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:47.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:47 smithi189 ceph-mon[34851]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:47.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:47 smithi159 ceph-mon[30472]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:48.216 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:26:47 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-4[45783]: 2024-09-19T07:26:47.833+0000 7f50668b8640 -1 osd.4 0 log_to_monitors true 2024-09-19T07:26:48.355 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:48 smithi203 ceph-mon[35335]: from='osd.4 [v2:172.21.15.189:6808/1036888180,v1:172.21.15.189:6809/1036888180]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T07:26:48.355 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:48 smithi203 ceph-mon[35335]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T07:26:48.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:48 smithi189 ceph-mon[34851]: from='osd.4 [v2:172.21.15.189:6808/1036888180,v1:172.21.15.189:6809/1036888180]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T07:26:48.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:48 smithi189 ceph-mon[34851]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T07:26:48.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:48 smithi159 ceph-mon[30472]: from='osd.4 [v2:172.21.15.189:6808/1036888180,v1:172.21.15.189:6809/1036888180]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T07:26:48.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:48 smithi159 ceph-mon[30472]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T07:26:48.941 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:26:48.966 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_3 2024-09-19T07:26:49.210 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:26:49.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:49 smithi203 ceph-mon[35335]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T07:26:49.498 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:49 smithi203 ceph-mon[35335]: from='osd.4 [v2:172.21.15.189:6808/1036888180,v1:172.21.15.189:6809/1036888180]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:26:49.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:49 smithi203 ceph-mon[35335]: osdmap e32: 5 total, 4 up, 5 in 2024-09-19T07:26:49.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:49 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:49.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:49 smithi203 ceph-mon[35335]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:26:49.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:49 smithi203 ceph-mon[35335]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:49.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:49 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:49.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:49 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:49.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:49 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:26:49.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:49 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:26:49.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:49 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:49.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:49 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:49.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:49 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:26:49.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:49 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:49.556 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:26:49 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-4[45783]: 2024-09-19T07:26:49.233+0000 7f506283b640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T07:26:49.556 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:26:49 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-4[45783]: 2024-09-19T07:26:49.239+0000 7f505e663640 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:26:49.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:49 smithi189 ceph-mon[34851]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T07:26:49.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:49 smithi189 ceph-mon[34851]: from='osd.4 [v2:172.21.15.189:6808/1036888180,v1:172.21.15.189:6809/1036888180]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:26:49.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:49 smithi189 ceph-mon[34851]: osdmap e32: 5 total, 4 up, 5 in 2024-09-19T07:26:49.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:49 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:49.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:49 smithi189 ceph-mon[34851]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:26:49.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:49 smithi189 ceph-mon[34851]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:49.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:49 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:49.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:49 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:49.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:49 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:26:49.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:49 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:26:49.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:49 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:49.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:49 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:49.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:49 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:26:49.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:49 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:49.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:49 smithi159 ceph-mon[30472]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T07:26:49.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:49 smithi159 ceph-mon[30472]: from='osd.4 [v2:172.21.15.189:6808/1036888180,v1:172.21.15.189:6809/1036888180]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:26:49.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:49 smithi159 ceph-mon[30472]: osdmap e32: 5 total, 4 up, 5 in 2024-09-19T07:26:49.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:49 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:49.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:49 smithi159 ceph-mon[30472]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:26:49.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:49 smithi159 ceph-mon[30472]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:49.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:49 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:49.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:49 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:49.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:49 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:26:49.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:49 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:26:49.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:49 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:49.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:49 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:49.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:49 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:26:49.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:49 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:26:49.733 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.708+0000 7f6a02975640 1 -- 172.21.15.203:0/2939277157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69fc102eb0 msgr2=0x7f69fc103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:26:49.733 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.708+0000 7f6a02975640 1 --2- 172.21.15.203:0/2939277157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69fc102eb0 0x7f69fc103330 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009930 tx=0x7f69e402a420 comp rx=0 tx=0).stop 2024-09-19T07:26:49.733 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.708+0000 7f6a02975640 1 -- 172.21.15.203:0/2939277157 shutdown_connections 2024-09-19T07:26:49.734 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.708+0000 7f6a02975640 1 --2- 172.21.15.203:0/2939277157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69fc103870 0x7f69fc10a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:49.734 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.708+0000 7f6a02975640 1 --2- 172.21.15.203:0/2939277157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69fc102eb0 0x7f69fc103330 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:49.734 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.708+0000 7f6a02975640 1 --2- 172.21.15.203:0/2939277157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69fc0ffed0 0x7f69fc1002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:49.734 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.708+0000 7f6a02975640 1 -- 172.21.15.203:0/2939277157 >> 172.21.15.203:0/2939277157 conn(0x7f69fc0fba40 msgr2=0x7f69fc0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:26:49.734 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.708+0000 7f6a02975640 1 -- 172.21.15.203:0/2939277157 shutdown_connections 2024-09-19T07:26:49.734 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.708+0000 7f6a02975640 1 -- 172.21.15.203:0/2939277157 wait complete. 2024-09-19T07:26:49.734 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.708+0000 7f6a02975640 1 Processor -- start 2024-09-19T07:26:49.734 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.708+0000 7f6a02975640 1 -- start start 2024-09-19T07:26:49.734 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.708+0000 7f6a02975640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69fc0ffed0 0x7f69fc19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:26:49.734 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.708+0000 7f6a02975640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69fc102eb0 0x7f69fc19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:26:49.734 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.708+0000 7f6a02975640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69fc103870 0x7f69fc1a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:26:49.734 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.708+0000 7f6a02975640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc10d890 con 0x7f69fc103870 2024-09-19T07:26:49.735 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.708+0000 7f6a02975640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc10d710 con 0x7f69fc0ffed0 2024-09-19T07:26:49.735 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.708+0000 7f6a02975640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc10da10 con 0x7f69fc102eb0 2024-09-19T07:26:49.735 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.709+0000 7f69f3fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69fc102eb0 0x7f69fc19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:26:49.735 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.709+0000 7f69f3fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69fc102eb0 0x7f69fc19a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:40646/0 (socket says 172.21.15.203:40646) 2024-09-19T07:26:49.735 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.709+0000 7f69f3fff640 1 -- 172.21.15.203:0/2508738497 learned_addr learned my addr 172.21.15.203:0/2508738497 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-19T07:26:49.735 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.709+0000 7f69fbfff640 1 --2- 172.21.15.203:0/2508738497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69fc0ffed0 0x7f69fc19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:26:49.735 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.709+0000 7f6a00f12640 1 --2- 172.21.15.203:0/2508738497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69fc103870 0x7f69fc1a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:26:49.735 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.709+0000 7f69f3fff640 1 -- 172.21.15.203:0/2508738497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69fc0ffed0 msgr2=0x7f69fc19a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:26:49.735 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.709+0000 7f69f3fff640 1 --2- 172.21.15.203:0/2508738497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69fc0ffed0 0x7f69fc19a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:49.735 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.709+0000 7f69f3fff640 1 -- 172.21.15.203:0/2508738497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69fc103870 msgr2=0x7f69fc1a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:26:49.735 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.709+0000 7f69f3fff640 1 --2- 172.21.15.203:0/2508738497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69fc103870 0x7f69fc1a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:26:49.735 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.709+0000 7f69f3fff640 1 -- 172.21.15.203:0/2508738497 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc1a1d90 con 0x7f69fc102eb0 2024-09-19T07:26:49.736 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.709+0000 7f69f3fff640 1 --2- 172.21.15.203:0/2508738497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69fc102eb0 0x7f69fc19a6a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009a50 tx=0x7f69e4004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:26:49.736 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.709+0000 7f69f9ffb640 1 -- 172.21.15.203:0/2508738497 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e403f030 con 0x7f69fc102eb0 2024-09-19T07:26:49.736 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.709+0000 7f69f9ffb640 1 -- 172.21.15.203:0/2508738497 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e4003ce0 con 0x7f69fc102eb0 2024-09-19T07:26:49.736 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.709+0000 7f6a02975640 1 -- 172.21.15.203:0/2508738497 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69fc1a2020 con 0x7f69fc102eb0 2024-09-19T07:26:49.736 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.709+0000 7f69f9ffb640 1 -- 172.21.15.203:0/2508738497 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4035070 con 0x7f69fc102eb0 2024-09-19T07:26:49.736 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.709+0000 7f6a02975640 1 -- 172.21.15.203:0/2508738497 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69fc1a2500 con 0x7f69fc102eb0 2024-09-19T07:26:49.736 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.710+0000 7f6a02975640 1 -- 172.21.15.203:0/2508738497 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69fc1a2760 con 0x7f69fc102eb0 2024-09-19T07:26:49.736 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.711+0000 7f69f9ffb640 1 -- 172.21.15.203:0/2508738497 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f69e4042030 con 0x7f69fc102eb0 2024-09-19T07:26:49.736 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.712+0000 7f69f9ffb640 1 --2- 172.21.15.203:0/2508738497 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f69e80a3270 0x7f69e80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:26:49.736 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.712+0000 7f69f9ffb640 1 -- 172.21.15.203:0/2508738497 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 3821+0+0 (secure 0 0 0) 0x7f69e40eeee0 con 0x7f69fc102eb0 2024-09-19T07:26:49.736 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.712+0000 7f69fbfff640 1 --2- 172.21.15.203:0/2508738497 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f69e80a3270 0x7f69e80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:26:49.736 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.713+0000 7f69fbfff640 1 --2- 172.21.15.203:0/2508738497 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f69e80a3270 0x7f69e80a5730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f69fc06a590 tx=0x7f69ec00a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:26:49.736 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.716+0000 7f69f9ffb640 1 -- 172.21.15.203:0/2508738497 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f69e40bcd90 con 0x7f69fc102eb0 2024-09-19T07:26:49.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:26:49.897+0000 7f6a02975640 1 -- 172.21.15.203:0/2508738497 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f69fc109df0 con 0x7f69e80a3270 2024-09-19T07:26:50.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:50 smithi203 ceph-mon[35335]: Detected new or changed devices on smithi189 2024-09-19T07:26:50.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:50 smithi203 ceph-mon[35335]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-19T07:26:50.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:50 smithi203 ceph-mon[35335]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-19T07:26:50.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:50 smithi203 ceph-mon[35335]: osdmap e33: 5 total, 4 up, 5 in 2024-09-19T07:26:50.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:50 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:50.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:50 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:50.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:50 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:26:50.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:50 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:26:50.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:50 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:50.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:50 smithi189 ceph-mon[34851]: Detected new or changed devices on smithi189 2024-09-19T07:26:50.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:50 smithi189 ceph-mon[34851]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-19T07:26:50.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:50 smithi189 ceph-mon[34851]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-19T07:26:50.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:50 smithi189 ceph-mon[34851]: osdmap e33: 5 total, 4 up, 5 in 2024-09-19T07:26:50.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:50 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:50.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:50 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:50.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:50 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:26:50.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:50 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:26:50.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:50 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:50.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:50 smithi159 ceph-mon[30472]: Detected new or changed devices on smithi189 2024-09-19T07:26:50.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:50 smithi159 ceph-mon[30472]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-19T07:26:50.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:50 smithi159 ceph-mon[30472]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-19T07:26:50.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:50 smithi159 ceph-mon[30472]: osdmap e33: 5 total, 4 up, 5 in 2024-09-19T07:26:50.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:50.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:50.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:26:50.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:26:50.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:26:51.349 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:51 smithi203 ceph-mon[35335]: purged_snaps scrub starts 2024-09-19T07:26:51.349 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:51 smithi203 ceph-mon[35335]: purged_snaps scrub ok 2024-09-19T07:26:51.349 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:51 smithi203 ceph-mon[35335]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:26:51.350 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:51 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:51.350 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:51 smithi203 ceph-mon[35335]: osd.4 [v2:172.21.15.189:6808/1036888180,v1:172.21.15.189:6809/1036888180] boot 2024-09-19T07:26:51.350 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:51 smithi203 ceph-mon[35335]: osdmap e34: 5 total, 5 up, 5 in 2024-09-19T07:26:51.350 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:51 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:51.350 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:51 smithi203 ceph-mon[35335]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:51.538 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:51 smithi159 ceph-mon[30472]: purged_snaps scrub starts 2024-09-19T07:26:51.539 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:51 smithi159 ceph-mon[30472]: purged_snaps scrub ok 2024-09-19T07:26:51.539 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:51 smithi159 ceph-mon[30472]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:26:51.539 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:51 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:51.539 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:51 smithi159 ceph-mon[30472]: osd.4 [v2:172.21.15.189:6808/1036888180,v1:172.21.15.189:6809/1036888180] boot 2024-09-19T07:26:51.539 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:51 smithi159 ceph-mon[30472]: osdmap e34: 5 total, 5 up, 5 in 2024-09-19T07:26:51.539 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:51 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:51.540 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:51 smithi159 ceph-mon[30472]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:51.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:51 smithi189 ceph-mon[34851]: purged_snaps scrub starts 2024-09-19T07:26:51.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:51 smithi189 ceph-mon[34851]: purged_snaps scrub ok 2024-09-19T07:26:51.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:51 smithi189 ceph-mon[34851]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:26:51.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:51 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:51.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:51 smithi189 ceph-mon[34851]: osd.4 [v2:172.21.15.189:6808/1036888180,v1:172.21.15.189:6809/1036888180] boot 2024-09-19T07:26:51.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:51 smithi189 ceph-mon[34851]: osdmap e34: 5 total, 5 up, 5 in 2024-09-19T07:26:51.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:51 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:26:51.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:51 smithi189 ceph-mon[34851]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T07:26:52.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:52 smithi203 ceph-mon[35335]: osdmap e35: 5 total, 5 up, 5 in 2024-09-19T07:26:52.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:52 smithi189 ceph-mon[34851]: osdmap e35: 5 total, 5 up, 5 in 2024-09-19T07:26:52.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:52 smithi159 ceph-mon[30472]: osdmap e35: 5 total, 5 up, 5 in 2024-09-19T07:26:53.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:53 smithi203 ceph-mon[35335]: from='client.? 172.21.15.203:0/3979891430' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86c48303-929d-42f4-bf48-cbb32649f95f"}]: dispatch 2024-09-19T07:26:53.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:53 smithi203 ceph-mon[35335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86c48303-929d-42f4-bf48-cbb32649f95f"}]: dispatch 2024-09-19T07:26:53.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:53 smithi203 ceph-mon[35335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86c48303-929d-42f4-bf48-cbb32649f95f"}]': finished 2024-09-19T07:26:53.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:53 smithi203 ceph-mon[35335]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T07:26:53.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:53 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:26:53.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:53 smithi203 ceph-mon[35335]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:26:53.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:53 smithi203 ceph-mon[35335]: from='client.? 172.21.15.203:0/566423707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:26:53.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:53 smithi189 ceph-mon[34851]: from='client.? 172.21.15.203:0/3979891430' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86c48303-929d-42f4-bf48-cbb32649f95f"}]: dispatch 2024-09-19T07:26:53.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:53 smithi189 ceph-mon[34851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86c48303-929d-42f4-bf48-cbb32649f95f"}]: dispatch 2024-09-19T07:26:53.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:53 smithi189 ceph-mon[34851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86c48303-929d-42f4-bf48-cbb32649f95f"}]': finished 2024-09-19T07:26:53.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:53 smithi189 ceph-mon[34851]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T07:26:53.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:53 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:26:53.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:53 smithi189 ceph-mon[34851]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:26:53.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:53 smithi189 ceph-mon[34851]: from='client.? 172.21.15.203:0/566423707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:26:53.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:53 smithi159 ceph-mon[30472]: from='client.? 172.21.15.203:0/3979891430' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86c48303-929d-42f4-bf48-cbb32649f95f"}]: dispatch 2024-09-19T07:26:53.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:53 smithi159 ceph-mon[30472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86c48303-929d-42f4-bf48-cbb32649f95f"}]: dispatch 2024-09-19T07:26:53.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:53 smithi159 ceph-mon[30472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86c48303-929d-42f4-bf48-cbb32649f95f"}]': finished 2024-09-19T07:26:53.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:53 smithi159 ceph-mon[30472]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T07:26:53.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:53 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:26:53.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:53 smithi159 ceph-mon[30472]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:26:53.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:53 smithi159 ceph-mon[30472]: from='client.? 172.21.15.203:0/566423707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:26:56.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:55 smithi203 ceph-mon[35335]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:26:56.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:55 smithi189 ceph-mon[34851]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:26:56.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:55 smithi159 ceph-mon[30472]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:26:58.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:57 smithi203 ceph-mon[35335]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:26:58.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:57 smithi189 ceph-mon[34851]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:26:58.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:57 smithi159 ceph-mon[30472]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:00.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:26:59 smithi203 ceph-mon[35335]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:00.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:26:59 smithi189 ceph-mon[34851]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:00.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:26:59 smithi159 ceph-mon[30472]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:01.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:00 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T07:27:01.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:00 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:01.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:00 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T07:27:01.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:00 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:01.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:00 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T07:27:01.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:00 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:01.916 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:01 smithi203 ceph-mon[35335]: Deploying daemon osd.5 on smithi203 2024-09-19T07:27:01.916 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:01 smithi203 ceph-mon[35335]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:02.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:01 smithi189 ceph-mon[34851]: Deploying daemon osd.5 on smithi203 2024-09-19T07:27:02.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:01 smithi189 ceph-mon[34851]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:02.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:01 smithi159 ceph-mon[30472]: Deploying daemon osd.5 on smithi203 2024-09-19T07:27:02.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:01 smithi159 ceph-mon[30472]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:03.963 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:03 smithi203 ceph-mon[35335]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:04.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:03 smithi189 ceph-mon[34851]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:04.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:03 smithi159 ceph-mon[30472]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:05.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:05 smithi203 ceph-mon[35335]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:06.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:05 smithi189 ceph-mon[34851]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:06.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:05 smithi159 ceph-mon[30472]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:08.026 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:07 smithi203 ceph-mon[35335]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:08.027 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:27:08.027 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:08.027 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:08.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:07 smithi189 ceph-mon[34851]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:08.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:27:08.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:08.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:08.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:07 smithi159 ceph-mon[30472]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:08.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:27:08.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:08.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:09.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:09 smithi203 ceph-mon[35335]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:09.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:09 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:09.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:09 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:09.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:09 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:09.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:09 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:27:09.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:09 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:10.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:09 smithi189 ceph-mon[34851]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:10.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:09 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:10.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:09 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:10.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:09 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:10.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:09 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:27:10.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:09 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:10.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:09 smithi159 ceph-mon[30472]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:10.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:09 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:10.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:09 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:10.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:09 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:10.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:09 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:27:10.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:09 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:10.982 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:27:10.982+0000 7f69f9ffb640 1 -- 172.21.15.203:0/2508738497 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f69fc109df0 con 0x7f69e80a3270 2024-09-19T07:27:10.982 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 5 on host 'smithi203' 2024-09-19T07:27:10.985 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:27:10.984+0000 7f6a02975640 1 -- 172.21.15.203:0/2508738497 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f69e80a3270 msgr2=0x7f69e80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:27:10.985 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:27:10.984+0000 7f6a02975640 1 --2- 172.21.15.203:0/2508738497 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f69e80a3270 0x7f69e80a5730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f69fc06a590 tx=0x7f69ec00a2d0 comp rx=0 tx=0).stop 2024-09-19T07:27:10.985 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:27:10.984+0000 7f6a02975640 1 -- 172.21.15.203:0/2508738497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69fc102eb0 msgr2=0x7f69fc19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:27:10.985 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:27:10.984+0000 7f6a02975640 1 --2- 172.21.15.203:0/2508738497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69fc102eb0 0x7f69fc19a6a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009a50 tx=0x7f69e4004360 comp rx=0 tx=0).stop 2024-09-19T07:27:10.985 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:27:10.984+0000 7f6a02975640 1 -- 172.21.15.203:0/2508738497 shutdown_connections 2024-09-19T07:27:10.985 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:27:10.984+0000 7f6a02975640 1 --2- 172.21.15.203:0/2508738497 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f69e80a3270 0x7f69e80a5730 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:10.985 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:27:10.984+0000 7f6a02975640 1 --2- 172.21.15.203:0/2508738497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69fc103870 0x7f69fc1a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:10.985 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:27:10.984+0000 7f6a02975640 1 --2- 172.21.15.203:0/2508738497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69fc102eb0 0x7f69fc19a6a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:10.985 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:27:10.984+0000 7f6a02975640 1 --2- 172.21.15.203:0/2508738497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69fc0ffed0 0x7f69fc19a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:10.985 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:27:10.984+0000 7f6a02975640 1 -- 172.21.15.203:0/2508738497 >> 172.21.15.203:0/2508738497 conn(0x7f69fc0fba40 msgr2=0x7f69fc1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:27:10.985 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:27:10.985+0000 7f6a02975640 1 -- 172.21.15.203:0/2508738497 shutdown_connections 2024-09-19T07:27:10.986 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:27:10.985+0000 7f6a02975640 1 -- 172.21.15.203:0/2508738497 wait complete. 2024-09-19T07:27:11.724 DEBUG:teuthology.orchestra.run.smithi203:osd.5> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T07:27:11.726 INFO:tasks.cephadm:Deploying osd.6 on smithi159 with /dev/vg_nvme/lv_2... 2024-09-19T07:27:11.727 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T07:27:11.963 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:11 smithi203 ceph-mon[35335]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:11.963 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:11 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:27:11.963 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:11 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:11.963 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:11 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:12.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:11 smithi189 ceph-mon[34851]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:12.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:11 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:27:12.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:11 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:12.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:11 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:12.100 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:27:12.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:11 smithi159 ceph-mon[30472]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:12.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:11 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:27:12.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:11 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:12.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:11 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:12.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:12 smithi203 ceph-mon[35335]: from='osd.5 [v2:172.21.15.203:6808/4212601512,v1:172.21.15.203:6809/4212601512]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T07:27:12.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:12 smithi203 ceph-mon[35335]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T07:27:12.786 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:27:12 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:27:12.522+0000 7fb717c35640 -1 osd.5 0 log_to_monitors true 2024-09-19T07:27:12.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:12 smithi159 ceph-mon[30472]: from='osd.5 [v2:172.21.15.203:6808/4212601512,v1:172.21.15.203:6809/4212601512]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T07:27:12.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:12 smithi159 ceph-mon[30472]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T07:27:13.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:12 smithi189 ceph-mon[34851]: from='osd.5 [v2:172.21.15.203:6808/4212601512,v1:172.21.15.203:6809/4212601512]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T07:27:13.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:12 smithi189 ceph-mon[34851]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T07:27:13.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:13 smithi159 ceph-mon[30472]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:13.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:13 smithi159 ceph-mon[30472]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T07:27:13.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:13 smithi159 ceph-mon[30472]: from='osd.5 [v2:172.21.15.203:6808/4212601512,v1:172.21.15.203:6809/4212601512]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:27:13.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:13 smithi159 ceph-mon[30472]: osdmap e37: 6 total, 5 up, 6 in 2024-09-19T07:27:13.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:13 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:27:13.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:13 smithi159 ceph-mon[30472]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:27:14.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:13 smithi203 ceph-mon[35335]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:14.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:13 smithi203 ceph-mon[35335]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T07:27:14.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:13 smithi203 ceph-mon[35335]: from='osd.5 [v2:172.21.15.203:6808/4212601512,v1:172.21.15.203:6809/4212601512]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:27:14.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:13 smithi203 ceph-mon[35335]: osdmap e37: 6 total, 5 up, 6 in 2024-09-19T07:27:14.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:13 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:27:14.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:13 smithi203 ceph-mon[35335]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:27:14.037 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:27:13 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:27:13.737+0000 7fb713bb8640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T07:27:14.037 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:27:13 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:27:13.744+0000 7fb70f9e0640 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:27:14.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:13 smithi189 ceph-mon[34851]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:13 smithi189 ceph-mon[34851]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T07:27:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:13 smithi189 ceph-mon[34851]: from='osd.5 [v2:172.21.15.203:6808/4212601512,v1:172.21.15.203:6809/4212601512]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:27:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:13 smithi189 ceph-mon[34851]: osdmap e37: 6 total, 5 up, 6 in 2024-09-19T07:27:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:13 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:27:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:13 smithi189 ceph-mon[34851]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:27:14.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:14 smithi203 ceph-mon[35335]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-19T07:27:14.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:14 smithi203 ceph-mon[35335]: osdmap e38: 6 total, 5 up, 6 in 2024-09-19T07:27:14.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:14 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:27:14.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:14 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:27:14.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:14 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:14.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:14 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:14.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:14 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:27:14.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:14 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:27:14.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:14 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:14.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:14 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:14.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:14 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:27:14.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:14 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:15.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:14 smithi189 ceph-mon[34851]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-19T07:27:15.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:14 smithi189 ceph-mon[34851]: osdmap e38: 6 total, 5 up, 6 in 2024-09-19T07:27:15.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:14 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:27:15.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:14 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:27:15.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:14 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:15.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:14 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:15.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:14 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:27:15.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:14 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:27:15.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:14 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:15.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:14 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:15.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:14 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:27:15.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:14 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:15.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:14 smithi159 ceph-mon[30472]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-19T07:27:15.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:14 smithi159 ceph-mon[30472]: osdmap e38: 6 total, 5 up, 6 in 2024-09-19T07:27:15.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:14 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:27:15.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:14 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:27:15.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:14 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:15.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:14 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:15.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:14 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:27:15.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:14 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:27:15.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:14 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:15.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:14 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:15.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:14 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:27:15.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:14 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:15.298 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:27:15.331 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi159:vg_nvme/lv_2 2024-09-19T07:27:15.706 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:27:16.006 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:15 smithi159 ceph-mon[30472]: purged_snaps scrub starts 2024-09-19T07:27:16.006 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:15 smithi159 ceph-mon[30472]: purged_snaps scrub ok 2024-09-19T07:27:16.006 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:15 smithi159 ceph-mon[30472]: Detected new or changed devices on smithi203 2024-09-19T07:27:16.006 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:15 smithi159 ceph-mon[30472]: Adjusting osd_memory_target on smithi203 to 8544M 2024-09-19T07:27:16.007 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:15 smithi159 ceph-mon[30472]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:16.007 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:15 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:27:16.007 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:15 smithi159 ceph-mon[30472]: osd.5 [v2:172.21.15.203:6808/4212601512,v1:172.21.15.203:6809/4212601512] boot 2024-09-19T07:27:16.007 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:15 smithi159 ceph-mon[30472]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T07:27:16.007 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:15 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:27:16.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:15 smithi203 ceph-mon[35335]: purged_snaps scrub starts 2024-09-19T07:27:16.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:15 smithi203 ceph-mon[35335]: purged_snaps scrub ok 2024-09-19T07:27:16.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:15 smithi203 ceph-mon[35335]: Detected new or changed devices on smithi203 2024-09-19T07:27:16.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:15 smithi203 ceph-mon[35335]: Adjusting osd_memory_target on smithi203 to 8544M 2024-09-19T07:27:16.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:15 smithi203 ceph-mon[35335]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:16.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:15 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:27:16.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:15 smithi203 ceph-mon[35335]: osd.5 [v2:172.21.15.203:6808/4212601512,v1:172.21.15.203:6809/4212601512] boot 2024-09-19T07:27:16.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:15 smithi203 ceph-mon[35335]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T07:27:16.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:15 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:27:16.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:15 smithi189 ceph-mon[34851]: purged_snaps scrub starts 2024-09-19T07:27:16.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:15 smithi189 ceph-mon[34851]: purged_snaps scrub ok 2024-09-19T07:27:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:15 smithi189 ceph-mon[34851]: Detected new or changed devices on smithi203 2024-09-19T07:27:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:15 smithi189 ceph-mon[34851]: Adjusting osd_memory_target on smithi203 to 8544M 2024-09-19T07:27:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:15 smithi189 ceph-mon[34851]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:27:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:15 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:27:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:15 smithi189 ceph-mon[34851]: osd.5 [v2:172.21.15.203:6808/4212601512,v1:172.21.15.203:6809/4212601512] boot 2024-09-19T07:27:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:15 smithi189 ceph-mon[34851]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T07:27:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:15 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:27:16.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.311+0000 7f0c22200640 1 -- 172.21.15.159:0/4145532523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c1c103e00 msgr2=0x7f0c1c104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:27:16.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.311+0000 7f0c22200640 1 --2- 172.21.15.159:0/4145532523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c1c103e00 0x7f0c1c104200 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009900 tx=0x7f0c0c02a410 comp rx=0 tx=0).stop 2024-09-19T07:27:16.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.311+0000 7f0c22200640 1 -- 172.21.15.159:0/4145532523 shutdown_connections 2024-09-19T07:27:16.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.311+0000 7f0c22200640 1 --2- 172.21.15.159:0/4145532523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c1c105920 0x7f0c1c10c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:16.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.311+0000 7f0c22200640 1 --2- 172.21.15.159:0/4145532523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c1c104f60 0x7f0c1c1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:16.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.311+0000 7f0c22200640 1 --2- 172.21.15.159:0/4145532523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c1c103e00 0x7f0c1c104200 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:16.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.311+0000 7f0c22200640 1 -- 172.21.15.159:0/4145532523 >> 172.21.15.159:0/4145532523 conn(0x7f0c1c0ff5b0 msgr2=0x7f0c1c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:27:16.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.311+0000 7f0c22200640 1 -- 172.21.15.159:0/4145532523 shutdown_connections 2024-09-19T07:27:16.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.311+0000 7f0c22200640 1 -- 172.21.15.159:0/4145532523 wait complete. 2024-09-19T07:27:16.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.312+0000 7f0c22200640 1 Processor -- start 2024-09-19T07:27:16.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.312+0000 7f0c22200640 1 -- start start 2024-09-19T07:27:16.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.312+0000 7f0c22200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c1c103e00 0x7f0c1c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:27:16.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.312+0000 7f0c22200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c1c104f60 0x7f0c1c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:27:16.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.312+0000 7f0c22200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c1c105920 0x7f0c1c1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:27:16.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.312+0000 7f0c22200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c10fa90 con 0x7f0c1c105920 2024-09-19T07:27:16.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.312+0000 7f0c22200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c10f910 con 0x7f0c1c103e00 2024-09-19T07:27:16.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.312+0000 7f0c22200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c10fc10 con 0x7f0c1c104f60 2024-09-19T07:27:16.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.313+0000 7f0c12a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c1c104f60 0x7f0c1c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:27:16.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.313+0000 7f0c1be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c1c105920 0x7f0c1c1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:27:16.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.313+0000 7f0c1b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c1c103e00 0x7f0c1c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:27:16.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.313+0000 7f0c12a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c1c104f60 0x7f0c1c19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:60654/0 (socket says 172.21.15.159:60654) 2024-09-19T07:27:16.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.313+0000 7f0c12a00640 1 -- 172.21.15.159:0/2473126351 learned_addr learned my addr 172.21.15.159:0/2473126351 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:27:16.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.313+0000 7f0c12a00640 1 -- 172.21.15.159:0/2473126351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c1c103e00 msgr2=0x7f0c1c19a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:27:16.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.313+0000 7f0c12a00640 1 --2- 172.21.15.159:0/2473126351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c1c103e00 0x7f0c1c19a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:16.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.313+0000 7f0c12a00640 1 -- 172.21.15.159:0/2473126351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c1c105920 msgr2=0x7f0c1c1a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:27:16.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.313+0000 7f0c12a00640 1 --2- 172.21.15.159:0/2473126351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c1c105920 0x7f0c1c1a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:16.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.313+0000 7f0c12a00640 1 -- 172.21.15.159:0/2473126351 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1c1a1ce0 con 0x7f0c1c104f60 2024-09-19T07:27:16.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.313+0000 7f0c1be00640 1 --2- 172.21.15.159:0/2473126351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c1c105920 0x7f0c1c1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:27:16.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.314+0000 7f0c12a00640 1 --2- 172.21.15.159:0/2473126351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c1c104f60 0x7f0c1c19a5f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400e9c0 tx=0x7f0c0400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:27:16.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.314+0000 7f0c18c00640 1 -- 172.21.15.159:0/2473126351 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0400cde0 con 0x7f0c1c104f60 2024-09-19T07:27:16.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.314+0000 7f0c18c00640 1 -- 172.21.15.159:0/2473126351 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c04010910 con 0x7f0c1c104f60 2024-09-19T07:27:16.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.314+0000 7f0c22200640 1 -- 172.21.15.159:0/2473126351 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1c1a1fd0 con 0x7f0c1c104f60 2024-09-19T07:27:16.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.314+0000 7f0c18c00640 1 -- 172.21.15.159:0/2473126351 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04018a00 con 0x7f0c1c104f60 2024-09-19T07:27:16.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.314+0000 7f0c22200640 1 -- 172.21.15.159:0/2473126351 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1c1a2480 con 0x7f0c1c104f60 2024-09-19T07:27:16.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.315+0000 7f0c22200640 1 -- 172.21.15.159:0/2473126351 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c1c10b1c0 con 0x7f0c1c104f60 2024-09-19T07:27:16.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.317+0000 7f0c18c00640 1 -- 172.21.15.159:0/2473126351 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f0c0401f020 con 0x7f0c1c104f60 2024-09-19T07:27:16.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.318+0000 7f0c18c00640 1 --2- 172.21.15.159:0/2473126351 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0c000a3270 0x7f0c000a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:27:16.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.318+0000 7f0c18c00640 1 -- 172.21.15.159:0/2473126351 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f0c040cecb0 con 0x7f0c1c104f60 2024-09-19T07:27:16.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.318+0000 7f0c1b400640 1 --2- 172.21.15.159:0/2473126351 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0c000a3270 0x7f0c000a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:27:16.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.319+0000 7f0c1b400640 1 --2- 172.21.15.159:0/2473126351 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0c000a3270 0x7f0c000a5730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0314e0 tx=0x7f0c0c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:27:16.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.322+0000 7f0c18c00640 1 -- 172.21.15.159:0/2473126351 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0c0409c7d0 con 0x7f0c1c104f60 2024-09-19T07:27:16.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:16.506+0000 7f0c22200640 1 -- 172.21.15.159:0/2473126351 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0c1c062630 con 0x7f0c000a3270 2024-09-19T07:27:17.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:16 smithi203 ceph-mon[35335]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T07:27:17.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:16 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:27:17.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:16 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:27:17.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:16 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:17.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:16 smithi189 ceph-mon[34851]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T07:27:17.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:27:17.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:27:17.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:17.100 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:16 smithi159 ceph-mon[30472]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T07:27:17.100 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:16 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:27:17.100 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:16 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:27:17.100 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:16 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:18.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:17 smithi203 ceph-mon[35335]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:27:18.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:17 smithi203 ceph-mon[35335]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:18.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:17 smithi203 ceph-mon[35335]: osdmap e41: 6 total, 6 up, 6 in 2024-09-19T07:27:18.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:17 smithi203 ceph-mon[35335]: osdmap e42: 6 total, 6 up, 6 in 2024-09-19T07:27:18.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:17 smithi189 ceph-mon[34851]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:27:18.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:17 smithi189 ceph-mon[34851]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:18.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:17 smithi189 ceph-mon[34851]: osdmap e41: 6 total, 6 up, 6 in 2024-09-19T07:27:18.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:17 smithi189 ceph-mon[34851]: osdmap e42: 6 total, 6 up, 6 in 2024-09-19T07:27:18.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:17 smithi159 ceph-mon[30472]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:27:18.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:17 smithi159 ceph-mon[30472]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:18.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:17 smithi159 ceph-mon[30472]: osdmap e41: 6 total, 6 up, 6 in 2024-09-19T07:27:18.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:17 smithi159 ceph-mon[30472]: osdmap e42: 6 total, 6 up, 6 in 2024-09-19T07:27:19.938 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:19 smithi159 ceph-mon[30472]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T07:27:19.938 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:19 smithi159 ceph-mon[30472]: osdmap e43: 6 total, 6 up, 6 in 2024-09-19T07:27:19.938 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:19 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/591747041' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0176ed1-c3a7-4d75-b213-fe88fab4441a"}]: dispatch 2024-09-19T07:27:19.938 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:19 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/591747041' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0176ed1-c3a7-4d75-b213-fe88fab4441a"}]': finished 2024-09-19T07:27:19.938 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:19 smithi159 ceph-mon[30472]: osdmap e44: 7 total, 6 up, 7 in 2024-09-19T07:27:19.938 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:19 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:27:20.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:19 smithi203 ceph-mon[35335]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T07:27:20.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:19 smithi203 ceph-mon[35335]: osdmap e43: 6 total, 6 up, 6 in 2024-09-19T07:27:20.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:19 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/591747041' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0176ed1-c3a7-4d75-b213-fe88fab4441a"}]: dispatch 2024-09-19T07:27:20.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:19 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/591747041' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0176ed1-c3a7-4d75-b213-fe88fab4441a"}]': finished 2024-09-19T07:27:20.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:19 smithi203 ceph-mon[35335]: osdmap e44: 7 total, 6 up, 7 in 2024-09-19T07:27:20.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:19 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:27:20.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:19 smithi189 ceph-mon[34851]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T07:27:20.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:19 smithi189 ceph-mon[34851]: osdmap e43: 6 total, 6 up, 6 in 2024-09-19T07:27:20.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:19 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/591747041' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0176ed1-c3a7-4d75-b213-fe88fab4441a"}]: dispatch 2024-09-19T07:27:20.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:19 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/591747041' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0176ed1-c3a7-4d75-b213-fe88fab4441a"}]': finished 2024-09-19T07:27:20.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:19 smithi189 ceph-mon[34851]: osdmap e44: 7 total, 6 up, 7 in 2024-09-19T07:27:20.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:19 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:27:21.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:20 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3114571750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:27:21.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:20 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3114571750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:27:21.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:20 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3114571750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:27:21.957 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:21 smithi189 ceph-mon[34851]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T07:27:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:21 smithi203 ceph-mon[35335]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T07:27:22.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:21 smithi159 ceph-mon[30472]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T07:27:24.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:23 smithi203 ceph-mon[35335]: pgmap v150: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-19T07:27:24.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:23 smithi189 ceph-mon[34851]: pgmap v150: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-19T07:27:24.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:23 smithi159 ceph-mon[30472]: pgmap v150: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-19T07:27:26.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:25 smithi203 ceph-mon[35335]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-19T07:27:26.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:25 smithi189 ceph-mon[34851]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-19T07:27:26.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:25 smithi159 ceph-mon[30472]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-19T07:27:27.941 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:27 smithi159 ceph-mon[30472]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:28.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:27 smithi203 ceph-mon[35335]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:28.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:27 smithi189 ceph-mon[34851]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:30.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:29 smithi203 ceph-mon[35335]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:30.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:29 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T07:27:30.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:29 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:30.038 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:29 smithi159 ceph-mon[30472]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:30.038 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:29 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T07:27:30.038 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:29 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:30.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:29 smithi189 ceph-mon[34851]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:30.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:29 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T07:27:30.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:29 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:31.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:30 smithi203 ceph-mon[35335]: Deploying daemon osd.6 on smithi159 2024-09-19T07:27:31.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:30 smithi189 ceph-mon[34851]: Deploying daemon osd.6 on smithi159 2024-09-19T07:27:31.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:30 smithi159 ceph-mon[30472]: Deploying daemon osd.6 on smithi159 2024-09-19T07:27:31.821 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:31 smithi159 ceph-mon[30472]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:32.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:31 smithi203 ceph-mon[35335]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:32.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:31 smithi189 ceph-mon[34851]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:33.702 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:33 smithi159 ceph-mon[30472]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:33 smithi203 ceph-mon[35335]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:34.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:33 smithi189 ceph-mon[34851]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:36.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:35 smithi203 ceph-mon[35335]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:36.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:35 smithi189 ceph-mon[34851]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:36.086 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:35 smithi159 ceph-mon[30472]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:38.031 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:37 smithi159 ceph-mon[30472]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:38.032 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:27:38.032 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:38.032 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:38.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:37 smithi203 ceph-mon[35335]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:38.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:27:38.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:38.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:38.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:37 smithi189 ceph-mon[34851]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:38.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:27:38.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:38.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:39 smithi203 ceph-mon[35335]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:40.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:39 smithi189 ceph-mon[34851]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:40.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:39 smithi159 ceph-mon[30472]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:42.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:41 smithi203 ceph-mon[35335]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:42.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:41 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:42.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:41 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:42.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:41 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:42.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:41 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:27:42.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:41 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:42.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:41 smithi189 ceph-mon[34851]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:42.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:41 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:42.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:41 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:42.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:41 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:42.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:41 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:27:42.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:41 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:42.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:41 smithi159 ceph-mon[30472]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:42.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:41 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:42.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:41 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:42.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:41 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:42.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:41 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:27:42.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:41 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:42.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:42.348+0000 7f0c18c00640 1 -- 172.21.15.159:0/2473126351 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0c1c062630 con 0x7f0c000a3270 2024-09-19T07:27:42.349 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 6 on host 'smithi159' 2024-09-19T07:27:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:42.350+0000 7f0c22200640 1 -- 172.21.15.159:0/2473126351 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0c000a3270 msgr2=0x7f0c000a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:27:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:42.350+0000 7f0c22200640 1 --2- 172.21.15.159:0/2473126351 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0c000a3270 0x7f0c000a5730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0314e0 tx=0x7f0c0c000bc0 comp rx=0 tx=0).stop 2024-09-19T07:27:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:42.350+0000 7f0c22200640 1 -- 172.21.15.159:0/2473126351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c1c104f60 msgr2=0x7f0c1c19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:27:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:42.350+0000 7f0c22200640 1 --2- 172.21.15.159:0/2473126351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c1c104f60 0x7f0c1c19a5f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400e9c0 tx=0x7f0c0400ee80 comp rx=0 tx=0).stop 2024-09-19T07:27:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:42.351+0000 7f0c22200640 1 -- 172.21.15.159:0/2473126351 shutdown_connections 2024-09-19T07:27:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:42.351+0000 7f0c22200640 1 --2- 172.21.15.159:0/2473126351 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0c000a3270 0x7f0c000a5730 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:42.351+0000 7f0c22200640 1 --2- 172.21.15.159:0/2473126351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c1c105920 0x7f0c1c1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:42.351+0000 7f0c22200640 1 --2- 172.21.15.159:0/2473126351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c1c104f60 0x7f0c1c19a5f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:42.351+0000 7f0c22200640 1 --2- 172.21.15.159:0/2473126351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c1c103e00 0x7f0c1c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:42.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:42.351+0000 7f0c22200640 1 -- 172.21.15.159:0/2473126351 >> 172.21.15.159:0/2473126351 conn(0x7f0c1c0ff5b0 msgr2=0x7f0c1c108390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:27:42.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:42.351+0000 7f0c22200640 1 -- 172.21.15.159:0/2473126351 shutdown_connections 2024-09-19T07:27:42.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:27:42.351+0000 7f0c22200640 1 -- 172.21.15.159:0/2473126351 wait complete. 2024-09-19T07:27:43.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:42 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:27:43.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:42 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:43.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:42 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:43.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:42 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:27:43.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:42 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:43.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:42 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:43.116 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:42 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:27:43.116 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:42 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:43.116 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:42 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:43.157 DEBUG:teuthology.orchestra.run.smithi159:osd.6> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T07:27:43.162 INFO:tasks.cephadm:Deploying osd.7 on smithi189 with /dev/vg_nvme/lv_2... 2024-09-19T07:27:43.162 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T07:27:43.419 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:27:43.698 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:27:43 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:27:43.403+0000 7f44e4834640 -1 osd.6 0 log_to_monitors true 2024-09-19T07:27:43.793 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:43 smithi189 ceph-mon[34851]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:43.794 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:43 smithi189 ceph-mon[34851]: from='osd.6 [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T07:27:44.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:43 smithi203 ceph-mon[35335]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:44.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:43 smithi203 ceph-mon[35335]: from='osd.6 [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T07:27:44.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:43 smithi159 ceph-mon[30472]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:44.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:43 smithi159 ceph-mon[30472]: from='osd.6 [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T07:27:45.441 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:45 smithi159 ceph-mon[30472]: from='osd.6 [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T07:27:45.441 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:45 smithi159 ceph-mon[30472]: osdmap e45: 7 total, 6 up, 7 in 2024-09-19T07:27:45.441 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:45 smithi159 ceph-mon[30472]: from='osd.6 [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-19T07:27:45.441 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:45 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:27:45.441 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:45 smithi159 ceph-mon[30472]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:45.442 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:27:45 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:27:45.367+0000 7f44df400640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T07:27:45.442 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:27:45 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:27:45.380+0000 7f44da400640 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:27:45.451 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:45 smithi189 ceph-mon[34851]: from='osd.6 [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T07:27:45.451 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:45 smithi189 ceph-mon[34851]: osdmap e45: 7 total, 6 up, 7 in 2024-09-19T07:27:45.451 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:45 smithi189 ceph-mon[34851]: from='osd.6 [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-19T07:27:45.451 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:45 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:27:45.452 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:45 smithi189 ceph-mon[34851]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:45.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:45 smithi203 ceph-mon[35335]: from='osd.6 [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T07:27:45.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:45 smithi203 ceph-mon[35335]: osdmap e45: 7 total, 6 up, 7 in 2024-09-19T07:27:45.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:45 smithi203 ceph-mon[35335]: from='osd.6 [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-19T07:27:45.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:45 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:27:45.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:45 smithi203 ceph-mon[35335]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:46.093 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:27:46.119 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_2 2024-09-19T07:27:46.376 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:27:46.541 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:46 smithi159 ceph-mon[30472]: from='osd.6 [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-19T07:27:46.541 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:46 smithi159 ceph-mon[30472]: osdmap e46: 7 total, 6 up, 7 in 2024-09-19T07:27:46.541 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:46 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:27:46.541 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:46 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:27:46.633 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:46 smithi189 ceph-mon[34851]: from='osd.6 [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-19T07:27:46.633 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:46 smithi189 ceph-mon[34851]: osdmap e46: 7 total, 6 up, 7 in 2024-09-19T07:27:46.633 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:46 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:27:46.633 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:46 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:27:46.687 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:46 smithi203 ceph-mon[35335]: from='osd.6 [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-19T07:27:46.688 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:46 smithi203 ceph-mon[35335]: osdmap e46: 7 total, 6 up, 7 in 2024-09-19T07:27:46.688 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:46 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:27:46.688 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:46 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:27:46.885 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.852+0000 7f6bd11c0640 1 -- 172.21.15.189:0/4250734708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bcc100b10 msgr2=0x7f6bcc100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:27:46.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.852+0000 7f6bd11c0640 1 --2- 172.21.15.189:0/4250734708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bcc100b10 0x7f6bcc100f10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009900 tx=0x7f6bbc02a410 comp rx=0 tx=0).stop 2024-09-19T07:27:46.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.852+0000 7f6bd11c0640 1 -- 172.21.15.189:0/4250734708 shutdown_connections 2024-09-19T07:27:46.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.852+0000 7f6bd11c0640 1 --2- 172.21.15.189:0/4250734708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bcc1026d0 0x7f6bcc108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:46.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.852+0000 7f6bd11c0640 1 --2- 172.21.15.189:0/4250734708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bcc101d10 0x7f6bcc102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:46.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.852+0000 7f6bd11c0640 1 --2- 172.21.15.189:0/4250734708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bcc100b10 0x7f6bcc100f10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:46.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.852+0000 7f6bd11c0640 1 -- 172.21.15.189:0/4250734708 >> 172.21.15.189:0/4250734708 conn(0x7f6bcc0fc2a0 msgr2=0x7f6bcc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:27:46.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.853+0000 7f6bd11c0640 1 -- 172.21.15.189:0/4250734708 shutdown_connections 2024-09-19T07:27:46.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.853+0000 7f6bd11c0640 1 -- 172.21.15.189:0/4250734708 wait complete. 2024-09-19T07:27:46.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.853+0000 7f6bd11c0640 1 Processor -- start 2024-09-19T07:27:46.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.853+0000 7f6bd11c0640 1 -- start start 2024-09-19T07:27:46.887 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.853+0000 7f6bd11c0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bcc100b10 0x7f6bcc197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:27:46.887 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.853+0000 7f6bd11c0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bcc101d10 0x7f6bcc1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:27:46.887 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.853+0000 7f6bd11c0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bcc1026d0 0x7f6bcc19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:27:46.887 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.853+0000 7f6bd11c0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc10d900 con 0x7f6bcc100b10 2024-09-19T07:27:46.887 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.853+0000 7f6bd11c0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc10d780 con 0x7f6bcc1026d0 2024-09-19T07:27:46.887 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.853+0000 7f6bd11c0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc10da80 con 0x7f6bcc101d10 2024-09-19T07:27:46.887 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.854+0000 7f6bcb59e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bcc1026d0 0x7f6bcc19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:27:46.887 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.854+0000 7f6bcb59e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bcc1026d0 0x7f6bcc19f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:49522/0 (socket says 172.21.15.189:49522) 2024-09-19T07:27:46.887 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.854+0000 7f6bcb59e640 1 -- 172.21.15.189:0/2924175340 learned_addr learned my addr 172.21.15.189:0/2924175340 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-19T07:27:46.887 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.854+0000 7f6bc3fff640 1 --2- 172.21.15.189:0/2924175340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bcc101d10 0x7f6bcc1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:27:46.887 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.854+0000 7f6bcad9d640 1 --2- 172.21.15.189:0/2924175340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bcc100b10 0x7f6bcc197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:27:46.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.854+0000 7f6bcb59e640 1 -- 172.21.15.189:0/2924175340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bcc101d10 msgr2=0x7f6bcc1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:27:46.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.854+0000 7f6bcb59e640 1 --2- 172.21.15.189:0/2924175340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bcc101d10 0x7f6bcc1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:46.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.854+0000 7f6bcb59e640 1 -- 172.21.15.189:0/2924175340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bcc100b10 msgr2=0x7f6bcc197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:27:46.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.854+0000 7f6bcb59e640 1 --2- 172.21.15.189:0/2924175340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bcc100b10 0x7f6bcc197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:27:46.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.854+0000 7f6bcb59e640 1 -- 172.21.15.189:0/2924175340 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc19fbe0 con 0x7f6bcc1026d0 2024-09-19T07:27:46.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.854+0000 7f6bcb59e640 1 --2- 172.21.15.189:0/2924175340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bcc1026d0 0x7f6bcc19f570 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009fd0 tx=0x7f6bc400def0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:27:46.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.854+0000 7f6bc8d99640 1 -- 172.21.15.189:0/2924175340 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4017070 con 0x7f6bcc1026d0 2024-09-19T07:27:46.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.854+0000 7f6bc8d99640 1 -- 172.21.15.189:0/2924175340 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bc4003e50 con 0x7f6bcc1026d0 2024-09-19T07:27:46.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.854+0000 7f6bd11c0640 1 -- 172.21.15.189:0/2924175340 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bcc19fed0 con 0x7f6bcc1026d0 2024-09-19T07:27:46.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.854+0000 7f6bc8d99640 1 -- 172.21.15.189:0/2924175340 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4010460 con 0x7f6bcc1026d0 2024-09-19T07:27:46.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.854+0000 7f6bd11c0640 1 -- 172.21.15.189:0/2924175340 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bcc1a0380 con 0x7f6bcc1026d0 2024-09-19T07:27:46.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.855+0000 7f6bd11c0640 1 -- 172.21.15.189:0/2924175340 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bcc107f50 con 0x7f6bcc1026d0 2024-09-19T07:27:46.889 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.857+0000 7f6bc8d99640 1 -- 172.21.15.189:0/2924175340 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f6bc4003970 con 0x7f6bcc1026d0 2024-09-19T07:27:46.889 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.857+0000 7f6bc8d99640 1 --2- 172.21.15.189:0/2924175340 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6bb80a32f0 0x7f6bb80a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:27:46.889 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.858+0000 7f6bc8d99640 1 -- 172.21.15.189:0/2924175340 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f6bc40ce870 con 0x7f6bcc1026d0 2024-09-19T07:27:46.889 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.858+0000 7f6bcad9d640 1 --2- 172.21.15.189:0/2924175340 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6bb80a32f0 0x7f6bb80a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:27:46.889 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.858+0000 7f6bcad9d640 1 --2- 172.21.15.189:0/2924175340 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6bb80a32f0 0x7f6bb80a57b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc006010 tx=0x7f6bbc000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:27:46.889 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:46.861+0000 7f6bc8d99640 1 -- 172.21.15.189:0/2924175340 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6bc409c320 con 0x7f6bcc1026d0 2024-09-19T07:27:47.039 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:27:47.039+0000 7f6bd11c0640 1 -- 172.21.15.189:0/2924175340 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6bcc062630 con 0x7f6bb80a32f0 2024-09-19T07:27:47.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: purged_snaps scrub starts 2024-09-19T07:27:47.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: purged_snaps scrub ok 2024-09-19T07:27:47.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: osd.6 [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578] boot 2024-09-19T07:27:47.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: osdmap e47: 7 total, 7 up, 7 in 2024-09-19T07:27:47.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:27:47.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:47.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:47.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:47.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:27:47.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:27:47.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:27:47.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:47.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:47.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:27:47.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:47.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:27:47.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:27:47.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:47.704 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: purged_snaps scrub starts 2024-09-19T07:27:47.705 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: purged_snaps scrub ok 2024-09-19T07:27:47.705 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: osd.6 [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578] boot 2024-09-19T07:27:47.705 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: osdmap e47: 7 total, 7 up, 7 in 2024-09-19T07:27:47.705 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:27:47.705 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:47.705 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:47.705 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:47.705 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:27:47.705 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:27:47.705 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:27:47.705 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:47.705 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:47.705 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:27:47.705 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:47.706 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:27:47.706 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:27:47.706 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:47.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: purged_snaps scrub starts 2024-09-19T07:27:47.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: purged_snaps scrub ok 2024-09-19T07:27:47.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: osd.6 [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578] boot 2024-09-19T07:27:47.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: osdmap e47: 7 total, 7 up, 7 in 2024-09-19T07:27:47.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:27:47.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:27:47.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:47.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:47.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:27:47.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:27:47.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:27:47.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:47.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:47.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:27:47.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:27:47.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:27:47.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:27:47.788 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:48.475 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:48 smithi189 ceph-mon[34851]: Detected new or changed devices on smithi159 2024-09-19T07:27:48.475 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:48 smithi189 ceph-mon[34851]: Adjusting osd_memory_target on smithi159 to 5696M 2024-09-19T07:27:48.476 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:48 smithi189 ceph-mon[34851]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:27:48.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:48 smithi159 ceph-mon[30472]: Detected new or changed devices on smithi159 2024-09-19T07:27:48.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:48 smithi159 ceph-mon[30472]: Adjusting osd_memory_target on smithi159 to 5696M 2024-09-19T07:27:48.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:48 smithi159 ceph-mon[30472]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:27:48.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:48 smithi203 ceph-mon[35335]: Detected new or changed devices on smithi159 2024-09-19T07:27:48.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:48 smithi203 ceph-mon[35335]: Adjusting osd_memory_target on smithi159 to 5696M 2024-09-19T07:27:48.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:48 smithi203 ceph-mon[35335]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:27:49.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:49 smithi159 ceph-mon[30472]: osdmap e48: 7 total, 7 up, 7 in 2024-09-19T07:27:49.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:49 smithi159 ceph-mon[30472]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:27:49.704 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:49 smithi189 ceph-mon[34851]: osdmap e48: 7 total, 7 up, 7 in 2024-09-19T07:27:49.704 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:49 smithi189 ceph-mon[34851]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:27:49.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:49 smithi203 ceph-mon[35335]: osdmap e48: 7 total, 7 up, 7 in 2024-09-19T07:27:49.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:49 smithi203 ceph-mon[35335]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:27:50.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:50 smithi203 ceph-mon[35335]: from='client.? 172.21.15.189:0/3171051543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "921bd59d-9415-4559-8e9f-035d336afd65"}]: dispatch 2024-09-19T07:27:50.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:50 smithi203 ceph-mon[35335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "921bd59d-9415-4559-8e9f-035d336afd65"}]: dispatch 2024-09-19T07:27:50.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:50 smithi203 ceph-mon[35335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "921bd59d-9415-4559-8e9f-035d336afd65"}]': finished 2024-09-19T07:27:50.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:50 smithi203 ceph-mon[35335]: osdmap e49: 8 total, 7 up, 8 in 2024-09-19T07:27:50.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:50 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:27:50.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:50 smithi203 ceph-mon[35335]: from='client.? 172.21.15.189:0/3400805758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:27:50.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:50 smithi189 ceph-mon[34851]: from='client.? 172.21.15.189:0/3171051543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "921bd59d-9415-4559-8e9f-035d336afd65"}]: dispatch 2024-09-19T07:27:50.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:50 smithi189 ceph-mon[34851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "921bd59d-9415-4559-8e9f-035d336afd65"}]: dispatch 2024-09-19T07:27:50.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:50 smithi189 ceph-mon[34851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "921bd59d-9415-4559-8e9f-035d336afd65"}]': finished 2024-09-19T07:27:50.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:50 smithi189 ceph-mon[34851]: osdmap e49: 8 total, 7 up, 8 in 2024-09-19T07:27:50.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:50 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:27:50.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:50 smithi189 ceph-mon[34851]: from='client.? 172.21.15.189:0/3400805758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:27:50.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:50 smithi159 ceph-mon[30472]: from='client.? 172.21.15.189:0/3171051543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "921bd59d-9415-4559-8e9f-035d336afd65"}]: dispatch 2024-09-19T07:27:50.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:50 smithi159 ceph-mon[30472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "921bd59d-9415-4559-8e9f-035d336afd65"}]: dispatch 2024-09-19T07:27:50.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:50 smithi159 ceph-mon[30472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "921bd59d-9415-4559-8e9f-035d336afd65"}]': finished 2024-09-19T07:27:50.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:50 smithi159 ceph-mon[30472]: osdmap e49: 8 total, 7 up, 8 in 2024-09-19T07:27:50.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:27:50.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:50 smithi159 ceph-mon[30472]: from='client.? 172.21.15.189:0/3400805758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:27:51.707 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:51 smithi189 ceph-mon[34851]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:27:51.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:51 smithi203 ceph-mon[35335]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:27:51.788 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:51 smithi159 ceph-mon[30472]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:27:54.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:53 smithi203 ceph-mon[35335]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:27:54.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:53 smithi189 ceph-mon[34851]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:27:54.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:53 smithi159 ceph-mon[30472]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:27:55.805 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:55 smithi189 ceph-mon[34851]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:27:56.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:55 smithi203 ceph-mon[35335]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:27:56.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:55 smithi159 ceph-mon[30472]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:27:57.984 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:57 smithi203 ceph-mon[35335]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:27:57.984 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:57 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T07:27:57.984 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:57 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:58.001 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:57 smithi189 ceph-mon[34851]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:27:58.001 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:57 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T07:27:58.002 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:57 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:58.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:57 smithi159 ceph-mon[30472]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:27:58.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T07:27:58.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:27:59.023 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:58 smithi189 ceph-mon[34851]: Deploying daemon osd.7 on smithi189 2024-09-19T07:27:59.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:58 smithi203 ceph-mon[35335]: Deploying daemon osd.7 on smithi189 2024-09-19T07:27:59.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:58 smithi159 ceph-mon[30472]: Deploying daemon osd.7 on smithi189 2024-09-19T07:27:59.956 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:27:59 smithi189 ceph-mon[34851]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:00.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:27:59 smithi203 ceph-mon[35335]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:00.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:27:59 smithi159 ceph-mon[30472]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:02.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:01 smithi203 ceph-mon[35335]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:02.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:01 smithi189 ceph-mon[34851]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:02.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:01 smithi159 ceph-mon[30472]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:04.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:03 smithi203 ceph-mon[35335]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:04.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:03 smithi189 ceph-mon[34851]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:04.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:03 smithi159 ceph-mon[30472]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:05.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:04 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:28:05.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:04 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:05.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:04 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:05.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:04 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:28:05.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:04 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:05.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:04 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:05.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:04 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:28:05.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:04 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:05.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:04 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:06.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:05 smithi203 ceph-mon[35335]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:06.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:05 smithi189 ceph-mon[34851]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:06.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:05 smithi159 ceph-mon[30472]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:07.590 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 7 on host 'smithi189' 2024-09-19T07:28:07.590 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:28:07.588+0000 7f6bc8d99640 1 -- 172.21.15.189:0/2924175340 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6bcc062630 con 0x7f6bb80a32f0 2024-09-19T07:28:07.592 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:28:07.590+0000 7f6bd11c0640 1 -- 172.21.15.189:0/2924175340 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6bb80a32f0 msgr2=0x7f6bb80a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:07.592 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:28:07.590+0000 7f6bd11c0640 1 --2- 172.21.15.189:0/2924175340 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6bb80a32f0 0x7f6bb80a57b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc006010 tx=0x7f6bbc000bc0 comp rx=0 tx=0).stop 2024-09-19T07:28:07.592 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:28:07.590+0000 7f6bd11c0640 1 -- 172.21.15.189:0/2924175340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bcc1026d0 msgr2=0x7f6bcc19f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:07.592 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:28:07.590+0000 7f6bd11c0640 1 --2- 172.21.15.189:0/2924175340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bcc1026d0 0x7f6bcc19f570 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009fd0 tx=0x7f6bc400def0 comp rx=0 tx=0).stop 2024-09-19T07:28:07.592 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:28:07.590+0000 7f6bd11c0640 1 -- 172.21.15.189:0/2924175340 shutdown_connections 2024-09-19T07:28:07.592 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:28:07.590+0000 7f6bd11c0640 1 --2- 172.21.15.189:0/2924175340 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6bb80a32f0 0x7f6bb80a57b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:07.592 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:28:07.590+0000 7f6bd11c0640 1 --2- 172.21.15.189:0/2924175340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bcc1026d0 0x7f6bcc19f570 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:07.592 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:28:07.590+0000 7f6bd11c0640 1 --2- 172.21.15.189:0/2924175340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bcc101d10 0x7f6bcc1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:07.592 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:28:07.590+0000 7f6bd11c0640 1 --2- 172.21.15.189:0/2924175340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bcc100b10 0x7f6bcc197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:07.592 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:28:07.590+0000 7f6bd11c0640 1 -- 172.21.15.189:0/2924175340 >> 172.21.15.189:0/2924175340 conn(0x7f6bcc0fc2a0 msgr2=0x7f6bcc105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:07.593 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:28:07.591+0000 7f6bd11c0640 1 -- 172.21.15.189:0/2924175340 shutdown_connections 2024-09-19T07:28:07.593 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:28:07.591+0000 7f6bd11c0640 1 -- 172.21.15.189:0/2924175340 wait complete. 2024-09-19T07:28:07.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:07.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:07.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:07.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:28:07.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:07.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:07 smithi189 ceph-mon[34851]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:07.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:28:07.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:07.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:07.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:07.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:07.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:07.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:28:07.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:07.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:07 smithi159 ceph-mon[30472]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:07.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:28:07.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:07.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:08.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:08.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:08.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:08.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:28:08.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:08.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:07 smithi203 ceph-mon[35335]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:08.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:28:08.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:08.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:08.447 DEBUG:teuthology.orchestra.run.smithi189:osd.7> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T07:28:08.450 INFO:tasks.cephadm:Deploying osd.8 on smithi203 with /dev/vg_nvme/lv_2... 2024-09-19T07:28:08.451 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T07:28:08.706 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:28:09.842 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:09 smithi189 ceph-mon[34851]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:09.843 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:28:09 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:28:09.820+0000 7efc99c74640 -1 osd.7 0 log_to_monitors true 2024-09-19T07:28:09.966 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:09 smithi203 ceph-mon[35335]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:10.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:09 smithi159 ceph-mon[30472]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:10.926 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:10 smithi189 ceph-mon[34851]: from='osd.7 [v2:172.21.15.189:6816/3936826456,v1:172.21.15.189:6817/3936826456]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T07:28:10.926 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:10 smithi189 ceph-mon[34851]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T07:28:11.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:10 smithi203 ceph-mon[35335]: from='osd.7 [v2:172.21.15.189:6816/3936826456,v1:172.21.15.189:6817/3936826456]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T07:28:11.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:10 smithi203 ceph-mon[35335]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T07:28:11.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:10 smithi159 ceph-mon[30472]: from='osd.7 [v2:172.21.15.189:6816/3936826456,v1:172.21.15.189:6817/3936826456]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T07:28:11.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:10 smithi159 ceph-mon[30472]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T07:28:11.378 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:28:11.405 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_2 2024-09-19T07:28:11.658 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:28:12.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:11 smithi203 ceph-mon[35335]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:12.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:11 smithi203 ceph-mon[35335]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T07:28:12.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:11 smithi203 ceph-mon[35335]: from='osd.7 [v2:172.21.15.189:6816/3936826456,v1:172.21.15.189:6817/3936826456]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:28:12.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:11 smithi203 ceph-mon[35335]: osdmap e50: 8 total, 7 up, 8 in 2024-09-19T07:28:12.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:11 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:28:12.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:11 smithi203 ceph-mon[35335]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:28:12.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:11 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:12.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:11 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:12.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:11 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:12.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:11 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:12.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:11 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:12.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:11 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:12.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:11 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:12.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:11 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:28:12.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:11 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:12.056 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:28:11 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:28:11.724+0000 7efc95bf7640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T07:28:12.056 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:28:11 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:28:11.735+0000 7efc91a1f640 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:28:12.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:11 smithi189 ceph-mon[34851]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:12.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:11 smithi189 ceph-mon[34851]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T07:28:12.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:11 smithi189 ceph-mon[34851]: from='osd.7 [v2:172.21.15.189:6816/3936826456,v1:172.21.15.189:6817/3936826456]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:28:12.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:11 smithi189 ceph-mon[34851]: osdmap e50: 8 total, 7 up, 8 in 2024-09-19T07:28:12.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:11 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:28:12.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:11 smithi189 ceph-mon[34851]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:28:12.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:11 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:12.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:11 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:12.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:11 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:12.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:11 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:12.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:11 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:12.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:11 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:12.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:11 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:12.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:11 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:28:12.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:11 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:12.121 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.098+0000 7f9dac559640 1 -- 172.21.15.203:0/1977298494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da4107970 msgr2=0x7f9da4109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:12.121 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.098+0000 7f9dac559640 1 --2- 172.21.15.203:0/1977298494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da4107970 0x7f9da4109d60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009ca0 tx=0x7f9d9c02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:28:12.121 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.098+0000 7f9dac559640 1 -- 172.21.15.203:0/1977298494 shutdown_connections 2024-09-19T07:28:12.121 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.098+0000 7f9dac559640 1 --2- 172.21.15.203:0/1977298494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da4107970 0x7f9da4109d60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:12.122 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.098+0000 7f9dac559640 1 --2- 172.21.15.203:0/1977298494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da4105040 0x7f9da4107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:12.122 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.098+0000 7f9dac559640 1 --2- 172.21.15.203:0/1977298494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9da4102710 0x7f9da4104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:12.122 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.098+0000 7f9dac559640 1 -- 172.21.15.203:0/1977298494 >> 172.21.15.203:0/1977298494 conn(0x7f9da40fc270 msgr2=0x7f9da40fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:12.122 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.098+0000 7f9dac559640 1 -- 172.21.15.203:0/1977298494 shutdown_connections 2024-09-19T07:28:12.122 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.098+0000 7f9dac559640 1 -- 172.21.15.203:0/1977298494 wait complete. 2024-09-19T07:28:12.122 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.098+0000 7f9dac559640 1 Processor -- start 2024-09-19T07:28:12.122 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9dac559640 1 -- start start 2024-09-19T07:28:12.122 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9dac559640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da4102710 0x7f9da4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:12.122 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9dac559640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da4105040 0x7f9da4198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:12.122 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9dac559640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9da4107970 0x7f9da419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:12.122 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9dac559640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9da410d980 con 0x7f9da4107970 2024-09-19T07:28:12.122 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9dac559640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9da410d800 con 0x7f9da4102710 2024-09-19T07:28:12.123 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9dac559640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9da410db00 con 0x7f9da4105040 2024-09-19T07:28:12.123 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9da9af4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da4105040 0x7f9da4198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:12.123 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9da9af4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da4105040 0x7f9da4198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:49308/0 (socket says 172.21.15.203:49308) 2024-09-19T07:28:12.123 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9da9af4640 1 -- 172.21.15.203:0/2268063717 learned_addr learned my addr 172.21.15.203:0/2268063717 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-19T07:28:12.123 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9daa2f5640 1 --2- 172.21.15.203:0/2268063717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da4102710 0x7f9da4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:12.123 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9daaaf6640 1 --2- 172.21.15.203:0/2268063717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9da4107970 0x7f9da419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:12.123 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9da9af4640 1 -- 172.21.15.203:0/2268063717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da4102710 msgr2=0x7f9da4198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:12.123 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9da9af4640 1 --2- 172.21.15.203:0/2268063717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da4102710 0x7f9da4198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:12.123 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9da9af4640 1 -- 172.21.15.203:0/2268063717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9da4107970 msgr2=0x7f9da419f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:12.123 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9da9af4640 1 --2- 172.21.15.203:0/2268063717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9da4107970 0x7f9da419f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:12.123 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9da9af4640 1 -- 172.21.15.203:0/2268063717 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da419fc30 con 0x7f9da4105040 2024-09-19T07:28:12.123 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9da9af4640 1 --2- 172.21.15.203:0/2268063717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da4105040 0x7f9da4198540 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9da000d740 tx=0x7f9da000dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:12.124 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9d9b7fe640 1 -- 172.21.15.203:0/2268063717 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0017070 con 0x7f9da4105040 2024-09-19T07:28:12.124 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.099+0000 7f9dac559640 1 -- 172.21.15.203:0/2268063717 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da419ff20 con 0x7f9da4105040 2024-09-19T07:28:12.124 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.100+0000 7f9d9b7fe640 1 -- 172.21.15.203:0/2268063717 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9da000de50 con 0x7f9da4105040 2024-09-19T07:28:12.124 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.100+0000 7f9d9b7fe640 1 -- 172.21.15.203:0/2268063717 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0004e00 con 0x7f9da4105040 2024-09-19T07:28:12.124 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.100+0000 7f9dac559640 1 -- 172.21.15.203:0/2268063717 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da41a0450 con 0x7f9da4105040 2024-09-19T07:28:12.124 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.101+0000 7f9dac559640 1 -- 172.21.15.203:0/2268063717 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da40fdd60 con 0x7f9da4105040 2024-09-19T07:28:12.124 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.102+0000 7f9d9b7fe640 1 -- 172.21.15.203:0/2268063717 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9da0004fa0 con 0x7f9da4105040 2024-09-19T07:28:12.124 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.102+0000 7f9d9b7fe640 1 --2- 172.21.15.203:0/2268063717 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9d900a3270 0x7f9d900a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:12.124 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.103+0000 7f9d9b7fe640 1 -- 172.21.15.203:0/2268063717 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5117+0+0 (secure 0 0 0) 0x7f9da00ce720 con 0x7f9da4105040 2024-09-19T07:28:12.124 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.103+0000 7f9daa2f5640 1 --2- 172.21.15.203:0/2268063717 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9d900a3270 0x7f9d900a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:12.124 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.103+0000 7f9daa2f5640 1 --2- 172.21.15.203:0/2268063717 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9d900a3270 0x7f9d900a5730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009890 tx=0x7f9d94008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:12.124 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.107+0000 7f9d9b7fe640 1 -- 172.21.15.203:0/2268063717 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9da009c0a0 con 0x7f9da4105040 2024-09-19T07:28:12.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:11 smithi159 ceph-mon[30472]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:12.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:11 smithi159 ceph-mon[30472]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T07:28:12.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:11 smithi159 ceph-mon[30472]: from='osd.7 [v2:172.21.15.189:6816/3936826456,v1:172.21.15.189:6817/3936826456]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:28:12.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:11 smithi159 ceph-mon[30472]: osdmap e50: 8 total, 7 up, 8 in 2024-09-19T07:28:12.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:11 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:28:12.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:11 smithi159 ceph-mon[30472]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:28:12.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:11 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:12.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:11 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:12.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:11 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:12.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:11 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:12.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:11 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:12.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:11 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:12.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:11 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:12.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:11 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:28:12.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:11 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:12.316 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:12.314+0000 7f9dac559640 1 -- 172.21.15.203:0/2268063717 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9da4062630 con 0x7f9d900a3270 2024-09-19T07:28:12.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:12 smithi203 ceph-mon[35335]: Detected new or changed devices on smithi189 2024-09-19T07:28:12.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:12 smithi203 ceph-mon[35335]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-19T07:28:12.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:12 smithi203 ceph-mon[35335]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-19T07:28:12.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:12 smithi203 ceph-mon[35335]: osdmap e51: 8 total, 7 up, 8 in 2024-09-19T07:28:12.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:28:12.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:28:12.983 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:28:12.984 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:28:12.984 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:12.984 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:12 smithi203 ceph-mon[35335]: osd.7 [v2:172.21.15.189:6816/3936826456,v1:172.21.15.189:6817/3936826456] boot 2024-09-19T07:28:12.984 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:12 smithi203 ceph-mon[35335]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T07:28:12.984 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:28:13.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:12 smithi189 ceph-mon[34851]: Detected new or changed devices on smithi189 2024-09-19T07:28:13.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:12 smithi189 ceph-mon[34851]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-19T07:28:13.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:12 smithi189 ceph-mon[34851]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-19T07:28:13.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:12 smithi189 ceph-mon[34851]: osdmap e51: 8 total, 7 up, 8 in 2024-09-19T07:28:13.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:28:13.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:28:13.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:28:13.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:28:13.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:13.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:12 smithi189 ceph-mon[34851]: osd.7 [v2:172.21.15.189:6816/3936826456,v1:172.21.15.189:6817/3936826456] boot 2024-09-19T07:28:13.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:12 smithi189 ceph-mon[34851]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T07:28:13.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:28:13.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:12 smithi159 ceph-mon[30472]: Detected new or changed devices on smithi189 2024-09-19T07:28:13.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:12 smithi159 ceph-mon[30472]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-19T07:28:13.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:12 smithi159 ceph-mon[30472]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-19T07:28:13.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:12 smithi159 ceph-mon[30472]: osdmap e51: 8 total, 7 up, 8 in 2024-09-19T07:28:13.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:28:13.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:28:13.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:28:13.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:28:13.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:13.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:12 smithi159 ceph-mon[30472]: osd.7 [v2:172.21.15.189:6816/3936826456,v1:172.21.15.189:6817/3936826456] boot 2024-09-19T07:28:13.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:12 smithi159 ceph-mon[30472]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T07:28:13.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:28:14.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:13 smithi203 ceph-mon[35335]: purged_snaps scrub starts 2024-09-19T07:28:14.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:13 smithi203 ceph-mon[35335]: purged_snaps scrub ok 2024-09-19T07:28:14.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:13 smithi203 ceph-mon[35335]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:28:14.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:13 smithi203 ceph-mon[35335]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:14.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:13 smithi203 ceph-mon[35335]: osdmap e53: 8 total, 8 up, 8 in 2024-09-19T07:28:14.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:13 smithi189 ceph-mon[34851]: purged_snaps scrub starts 2024-09-19T07:28:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:13 smithi189 ceph-mon[34851]: purged_snaps scrub ok 2024-09-19T07:28:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:13 smithi189 ceph-mon[34851]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:28:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:13 smithi189 ceph-mon[34851]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:13 smithi189 ceph-mon[34851]: osdmap e53: 8 total, 8 up, 8 in 2024-09-19T07:28:14.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:13 smithi159 ceph-mon[30472]: purged_snaps scrub starts 2024-09-19T07:28:14.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:13 smithi159 ceph-mon[30472]: purged_snaps scrub ok 2024-09-19T07:28:14.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:13 smithi159 ceph-mon[30472]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:28:14.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:13 smithi159 ceph-mon[30472]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:28:14.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:13 smithi159 ceph-mon[30472]: osdmap e53: 8 total, 8 up, 8 in 2024-09-19T07:28:15.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:15 smithi203 ceph-mon[35335]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:15.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:15 smithi203 ceph-mon[35335]: osdmap e54: 8 total, 8 up, 8 in 2024-09-19T07:28:15.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:15 smithi203 ceph-mon[35335]: from='client.? 172.21.15.203:0/4294493383' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3"}]: dispatch 2024-09-19T07:28:15.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:15 smithi203 ceph-mon[35335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3"}]: dispatch 2024-09-19T07:28:15.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:15 smithi203 ceph-mon[35335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3"}]': finished 2024-09-19T07:28:15.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:15 smithi203 ceph-mon[35335]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T07:28:15.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:15 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:15.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:15 smithi203 ceph-mon[35335]: from='client.? 172.21.15.203:0/712186423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:28:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:15 smithi189 ceph-mon[34851]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:15 smithi189 ceph-mon[34851]: osdmap e54: 8 total, 8 up, 8 in 2024-09-19T07:28:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:15 smithi189 ceph-mon[34851]: from='client.? 172.21.15.203:0/4294493383' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3"}]: dispatch 2024-09-19T07:28:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:15 smithi189 ceph-mon[34851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3"}]: dispatch 2024-09-19T07:28:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:15 smithi189 ceph-mon[34851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3"}]': finished 2024-09-19T07:28:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:15 smithi189 ceph-mon[34851]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T07:28:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:15 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:15 smithi189 ceph-mon[34851]: from='client.? 172.21.15.203:0/712186423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:28:16.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:15 smithi159 ceph-mon[30472]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:16.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:15 smithi159 ceph-mon[30472]: osdmap e54: 8 total, 8 up, 8 in 2024-09-19T07:28:16.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:15 smithi159 ceph-mon[30472]: from='client.? 172.21.15.203:0/4294493383' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3"}]: dispatch 2024-09-19T07:28:16.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:15 smithi159 ceph-mon[30472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3"}]: dispatch 2024-09-19T07:28:16.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:15 smithi159 ceph-mon[30472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3"}]': finished 2024-09-19T07:28:16.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:15 smithi159 ceph-mon[30472]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T07:28:16.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:15 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:16.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:15 smithi159 ceph-mon[30472]: from='client.? 172.21.15.203:0/712186423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:28:17.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:16 smithi159 ceph-mon[30472]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T07:28:17.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:16 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:17.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:16 smithi203 ceph-mon[35335]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T07:28:17.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:16 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:17.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:16 smithi189 ceph-mon[34851]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T07:28:17.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:16 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:18.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:17 smithi159 ceph-mon[30472]: pgmap v189: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:18.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:17 smithi159 ceph-mon[30472]: osdmap e57: 9 total, 8 up, 9 in 2024-09-19T07:28:18.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:17 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:18.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:17 smithi203 ceph-mon[35335]: pgmap v189: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:18.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:17 smithi203 ceph-mon[35335]: osdmap e57: 9 total, 8 up, 9 in 2024-09-19T07:28:18.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:17 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:18.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:17 smithi189 ceph-mon[34851]: pgmap v189: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:18.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:17 smithi189 ceph-mon[34851]: osdmap e57: 9 total, 8 up, 9 in 2024-09-19T07:28:18.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:17 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:19.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:19 smithi203 ceph-mon[35335]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:20.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:19 smithi159 ceph-mon[30472]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:20.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:19 smithi189 ceph-mon[34851]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:22.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:21 smithi159 ceph-mon[30472]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:22.189 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:21 smithi203 ceph-mon[35335]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:22.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:21 smithi189 ceph-mon[34851]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:24.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:23 smithi159 ceph-mon[30472]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:24.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:23 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T07:28:24.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:23 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:24.193 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:23 smithi203 ceph-mon[35335]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:24.193 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:23 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T07:28:24.193 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:23 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:24.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:23 smithi189 ceph-mon[34851]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:24.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:23 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T07:28:24.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:23 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:24.981 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:24 smithi203 ceph-mon[35335]: Deploying daemon osd.8 on smithi203 2024-09-19T07:28:25.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:24 smithi159 ceph-mon[30472]: Deploying daemon osd.8 on smithi203 2024-09-19T07:28:25.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:24 smithi189 ceph-mon[34851]: Deploying daemon osd.8 on smithi203 2024-09-19T07:28:26.210 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:25 smithi203 ceph-mon[35335]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:26.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:25 smithi189 ceph-mon[34851]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:26.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:25 smithi159 ceph-mon[30472]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:27.158 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:26 smithi203 ceph-mon[35335]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:27.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:26 smithi189 ceph-mon[34851]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:27.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:26 smithi159 ceph-mon[30472]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:29.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:29 smithi203 ceph-mon[35335]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:30.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:29 smithi189 ceph-mon[34851]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:30.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:29 smithi159 ceph-mon[30472]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:30.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:28:30.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:30.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:28:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:31.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:28:31.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:31.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:32.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:31 smithi203 ceph-mon[35335]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:32.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:31 smithi189 ceph-mon[34851]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:32.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:31 smithi159 ceph-mon[30472]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:33.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:33.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:33.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:33.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:28:33.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:33.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:33 smithi189 ceph-mon[34851]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:33.811 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:33.811+0000 7f9d9b7fe640 1 -- 172.21.15.203:0/2268063717 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9da4062630 con 0x7f9d900a3270 2024-09-19T07:28:33.811 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 8 on host 'smithi203' 2024-09-19T07:28:33.814 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:33.814+0000 7f9dac559640 1 -- 172.21.15.203:0/2268063717 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9d900a3270 msgr2=0x7f9d900a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:33.814 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:33.814+0000 7f9dac559640 1 --2- 172.21.15.203:0/2268063717 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9d900a3270 0x7f9d900a5730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009890 tx=0x7f9d94008040 comp rx=0 tx=0).stop 2024-09-19T07:28:33.814 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:33.814+0000 7f9dac559640 1 -- 172.21.15.203:0/2268063717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da4105040 msgr2=0x7f9da4198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:33.814 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:33.814+0000 7f9dac559640 1 --2- 172.21.15.203:0/2268063717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da4105040 0x7f9da4198540 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9da000d740 tx=0x7f9da000dc00 comp rx=0 tx=0).stop 2024-09-19T07:28:33.814 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:33.814+0000 7f9dac559640 1 -- 172.21.15.203:0/2268063717 shutdown_connections 2024-09-19T07:28:33.815 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:33.814+0000 7f9dac559640 1 --2- 172.21.15.203:0/2268063717 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9d900a3270 0x7f9d900a5730 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:33.815 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:33.814+0000 7f9dac559640 1 --2- 172.21.15.203:0/2268063717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9da4107970 0x7f9da419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:33.815 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:33.814+0000 7f9dac559640 1 --2- 172.21.15.203:0/2268063717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da4105040 0x7f9da4198540 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:33.815 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:33.814+0000 7f9dac559640 1 --2- 172.21.15.203:0/2268063717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da4102710 0x7f9da4198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:33.815 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:33.814+0000 7f9dac559640 1 -- 172.21.15.203:0/2268063717 >> 172.21.15.203:0/2268063717 conn(0x7f9da40fc270 msgr2=0x7f9da4103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:33.815 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:33.814+0000 7f9dac559640 1 -- 172.21.15.203:0/2268063717 shutdown_connections 2024-09-19T07:28:33.815 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:28:33.814+0000 7f9dac559640 1 -- 172.21.15.203:0/2268063717 wait complete. 2024-09-19T07:28:33.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:33.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:33.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:33.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:28:33.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:33.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:33 smithi159 ceph-mon[30472]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:28:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:33 smithi203 ceph-mon[35335]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:34.449 DEBUG:teuthology.orchestra.run.smithi203:osd.8> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.8.service 2024-09-19T07:28:34.451 INFO:tasks.cephadm:Deploying osd.9 on smithi159 with /dev/vg_nvme/lv_1... 2024-09-19T07:28:34.451 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T07:28:34.681 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:34 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:28:34.681 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:34 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:34.681 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:34 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:34.829 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:28:34.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:34 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:28:34.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:34 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:34.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:34 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:35.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:34 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:28:35.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:34 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:35.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:34 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:35.712 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:35 smithi203 ceph-mon[35335]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:36.014 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:28:35 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:28:35.811+0000 7fdde1b86640 -1 osd.8 0 log_to_monitors true 2024-09-19T07:28:36.018 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:35 smithi159 ceph-mon[30472]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:36.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:35 smithi189 ceph-mon[34851]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:36.950 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:36 smithi159 ceph-mon[30472]: from='osd.8 [v2:172.21.15.203:6816/4062926645,v1:172.21.15.203:6817/4062926645]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T07:28:36.950 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:36 smithi159 ceph-mon[30472]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T07:28:37.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:36 smithi203 ceph-mon[35335]: from='osd.8 [v2:172.21.15.203:6816/4062926645,v1:172.21.15.203:6817/4062926645]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T07:28:37.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:36 smithi203 ceph-mon[35335]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T07:28:37.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:36 smithi189 ceph-mon[34851]: from='osd.8 [v2:172.21.15.203:6816/4062926645,v1:172.21.15.203:6817/4062926645]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T07:28:37.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:36 smithi189 ceph-mon[34851]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T07:28:37.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:37 smithi203 ceph-mon[35335]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T07:28:37.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:37 smithi203 ceph-mon[35335]: osdmap e58: 9 total, 8 up, 9 in 2024-09-19T07:28:37.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:37.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:37 smithi203 ceph-mon[35335]: from='osd.8 [v2:172.21.15.203:6816/4062926645,v1:172.21.15.203:6817/4062926645]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:28:37.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:37 smithi203 ceph-mon[35335]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:28:37.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:37 smithi203 ceph-mon[35335]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:37.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:37.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:37.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:37.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:37.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:37.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:37.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:37.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:28:37.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:37.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:37 smithi159 ceph-mon[30472]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T07:28:37.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:37 smithi159 ceph-mon[30472]: osdmap e58: 9 total, 8 up, 9 in 2024-09-19T07:28:37.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:37.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:37 smithi159 ceph-mon[30472]: from='osd.8 [v2:172.21.15.203:6816/4062926645,v1:172.21.15.203:6817/4062926645]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:28:37.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:37 smithi159 ceph-mon[30472]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:28:37.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:37 smithi159 ceph-mon[30472]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:37.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:37.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:37.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:37.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:37.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:37.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:37.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:37.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:28:37.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:37 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:38.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:37 smithi189 ceph-mon[34851]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T07:28:38.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:37 smithi189 ceph-mon[34851]: osdmap e58: 9 total, 8 up, 9 in 2024-09-19T07:28:38.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:38.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:37 smithi189 ceph-mon[34851]: from='osd.8 [v2:172.21.15.203:6816/4062926645,v1:172.21.15.203:6817/4062926645]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:28:38.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:37 smithi189 ceph-mon[34851]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:28:38.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:37 smithi189 ceph-mon[34851]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:38.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:38.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:38.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:38.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:38.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:28:38.060 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:38.060 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:38.060 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:28:38.060 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:28:38.487 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:28:38.519 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi159:vg_nvme/lv_1 2024-09-19T07:28:38.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:28:38 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:28:38.047+0000 7fddddb09640 -1 osd.8 0 waiting for initial osdmap 2024-09-19T07:28:38.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:28:38 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:28:38.057+0000 7fddd5929640 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:28:38.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:38 smithi159 ceph-mon[30472]: Detected new or changed devices on smithi203 2024-09-19T07:28:38.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:38 smithi159 ceph-mon[30472]: Adjusting osd_memory_target on smithi203 to 5696M 2024-09-19T07:28:38.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:38 smithi159 ceph-mon[30472]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-19T07:28:38.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:38 smithi159 ceph-mon[30472]: osdmap e59: 9 total, 8 up, 9 in 2024-09-19T07:28:38.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:38 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:38.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:38 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:38.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:38 smithi159 ceph-mon[30472]: osdmap e60: 9 total, 8 up, 9 in 2024-09-19T07:28:38.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:38 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:38.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:38 smithi159 ceph-mon[30472]: from='osd.8 ' entity='osd.8' 2024-09-19T07:28:38.912 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:28:39.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:38 smithi203 ceph-mon[35335]: Detected new or changed devices on smithi203 2024-09-19T07:28:39.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:38 smithi203 ceph-mon[35335]: Adjusting osd_memory_target on smithi203 to 5696M 2024-09-19T07:28:39.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:38 smithi203 ceph-mon[35335]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-19T07:28:39.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:38 smithi203 ceph-mon[35335]: osdmap e59: 9 total, 8 up, 9 in 2024-09-19T07:28:39.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:38 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:39.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:38 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:39.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:38 smithi203 ceph-mon[35335]: osdmap e60: 9 total, 8 up, 9 in 2024-09-19T07:28:39.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:38 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:39.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:38 smithi203 ceph-mon[35335]: from='osd.8 ' entity='osd.8' 2024-09-19T07:28:39.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:38 smithi189 ceph-mon[34851]: Detected new or changed devices on smithi203 2024-09-19T07:28:39.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:38 smithi189 ceph-mon[34851]: Adjusting osd_memory_target on smithi203 to 5696M 2024-09-19T07:28:39.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:38 smithi189 ceph-mon[34851]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-19T07:28:39.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:38 smithi189 ceph-mon[34851]: osdmap e59: 9 total, 8 up, 9 in 2024-09-19T07:28:39.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:38 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:39.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:38 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:39.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:38 smithi189 ceph-mon[34851]: osdmap e60: 9 total, 8 up, 9 in 2024-09-19T07:28:39.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:38 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:39.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:38 smithi189 ceph-mon[34851]: from='osd.8 ' entity='osd.8' 2024-09-19T07:28:39.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.590+0000 7f015a200640 1 -- 172.21.15.159:0/3909915463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0154105920 msgr2=0x7f015410c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:39.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.590+0000 7f015a200640 1 --2- 172.21.15.159:0/3909915463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0154105920 0x7f015410c240 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f014c009900 tx=0x7f014c02a3f0 comp rx=0 tx=0).stop 2024-09-19T07:28:39.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.591+0000 7f015a200640 1 -- 172.21.15.159:0/3909915463 shutdown_connections 2024-09-19T07:28:39.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.591+0000 7f015a200640 1 --2- 172.21.15.159:0/3909915463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0154105920 0x7f015410c240 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:39.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.591+0000 7f015a200640 1 --2- 172.21.15.159:0/3909915463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0154104f60 0x7f01541053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:39.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.591+0000 7f015a200640 1 --2- 172.21.15.159:0/3909915463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0154103e00 0x7f0154104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:39.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.591+0000 7f015a200640 1 -- 172.21.15.159:0/3909915463 >> 172.21.15.159:0/3909915463 conn(0x7f01540ff5b0 msgr2=0x7f01541019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:39.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.591+0000 7f015a200640 1 -- 172.21.15.159:0/3909915463 shutdown_connections 2024-09-19T07:28:39.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.591+0000 7f015a200640 1 -- 172.21.15.159:0/3909915463 wait complete. 2024-09-19T07:28:39.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.592+0000 7f015a200640 1 Processor -- start 2024-09-19T07:28:39.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.592+0000 7f015a200640 1 -- start start 2024-09-19T07:28:39.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.592+0000 7f015a200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0154103e00 0x7f015419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:39.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.592+0000 7f015a200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0154104f60 0x7f015419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:39.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.592+0000 7f015a200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0154105920 0x7f01541a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:39.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.592+0000 7f015a200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f015410fa90 con 0x7f0154103e00 2024-09-19T07:28:39.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.592+0000 7f015a200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f015410f910 con 0x7f0154104f60 2024-09-19T07:28:39.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.592+0000 7f015a200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f015410fc10 con 0x7f0154105920 2024-09-19T07:28:39.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.592+0000 7f0153400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0154103e00 0x7f015419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:39.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.592+0000 7f014aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0154104f60 0x7f015419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:39.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.592+0000 7f014aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0154104f60 0x7f015419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:37574/0 (socket says 172.21.15.159:37574) 2024-09-19T07:28:39.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.592+0000 7f014aa00640 1 -- 172.21.15.159:0/2446694327 learned_addr learned my addr 172.21.15.159:0/2446694327 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:28:39.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.593+0000 7f0153e00640 1 --2- 172.21.15.159:0/2446694327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0154105920 0x7f01541a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:39.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.593+0000 7f014aa00640 1 -- 172.21.15.159:0/2446694327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0154105920 msgr2=0x7f01541a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:39.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.593+0000 7f014aa00640 1 --2- 172.21.15.159:0/2446694327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0154105920 0x7f01541a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:39.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.593+0000 7f014aa00640 1 -- 172.21.15.159:0/2446694327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0154103e00 msgr2=0x7f015419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:39.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.593+0000 7f014aa00640 1 --2- 172.21.15.159:0/2446694327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0154103e00 0x7f015419a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:39.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.593+0000 7f014aa00640 1 -- 172.21.15.159:0/2446694327 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01541a1cb0 con 0x7f0154104f60 2024-09-19T07:28:39.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.593+0000 7f0153400640 1 --2- 172.21.15.159:0/2446694327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0154103e00 0x7f015419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:28:39.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.593+0000 7f014aa00640 1 --2- 172.21.15.159:0/2446694327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0154104f60 0x7f015419a5c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f013c00ebc0 tx=0x7f013c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:39.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.593+0000 7f0150c00640 1 -- 172.21.15.159:0/2446694327 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c003b70 con 0x7f0154104f60 2024-09-19T07:28:39.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.594+0000 7f0150c00640 1 -- 172.21.15.159:0/2446694327 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f013c003d10 con 0x7f0154104f60 2024-09-19T07:28:39.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.594+0000 7f015a200640 1 -- 172.21.15.159:0/2446694327 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01541a1fa0 con 0x7f0154104f60 2024-09-19T07:28:39.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.594+0000 7f0150c00640 1 -- 172.21.15.159:0/2446694327 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c019980 con 0x7f0154104f60 2024-09-19T07:28:39.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.594+0000 7f015a200640 1 -- 172.21.15.159:0/2446694327 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01541a2450 con 0x7f0154104f60 2024-09-19T07:28:39.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.595+0000 7f015a200640 1 -- 172.21.15.159:0/2446694327 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015410b120 con 0x7f0154104f60 2024-09-19T07:28:39.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.597+0000 7f0150c00640 1 -- 172.21.15.159:0/2446694327 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f013c004b60 con 0x7f0154104f60 2024-09-19T07:28:39.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.598+0000 7f0150c00640 1 --2- 172.21.15.159:0/2446694327 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f01380a3270 0x7f01380a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:39.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.598+0000 7f0150c00640 1 -- 172.21.15.159:0/2446694327 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5862+0+0 (secure 0 0 0) 0x7f013c0cfad0 con 0x7f0154104f60 2024-09-19T07:28:39.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.598+0000 7f0153400640 1 --2- 172.21.15.159:0/2446694327 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f01380a3270 0x7f01380a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:39.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.599+0000 7f0153400640 1 --2- 172.21.15.159:0/2446694327 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f01380a3270 0x7f01380a5730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0144009fc0 tx=0x7f0144005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:39.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.600+0000 7f0150c00640 1 -- 172.21.15.159:0/2446694327 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f013c09d0e0 con 0x7f0154104f60 2024-09-19T07:28:39.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:28:39.794+0000 7f015a200640 1 -- 172.21.15.159:0/2446694327 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0154062630 con 0x7f01380a3270 2024-09-19T07:28:39.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:39 smithi159 ceph-mon[30472]: purged_snaps scrub starts 2024-09-19T07:28:39.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:39 smithi159 ceph-mon[30472]: purged_snaps scrub ok 2024-09-19T07:28:39.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:39 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:39.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:39 smithi159 ceph-mon[30472]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:39.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:39 smithi159 ceph-mon[30472]: osd.8 [v2:172.21.15.203:6816/4062926645,v1:172.21.15.203:6817/4062926645] boot 2024-09-19T07:28:39.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:39 smithi159 ceph-mon[30472]: osdmap e61: 9 total, 9 up, 9 in 2024-09-19T07:28:39.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:39 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:39 smithi203 ceph-mon[35335]: purged_snaps scrub starts 2024-09-19T07:28:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:39 smithi203 ceph-mon[35335]: purged_snaps scrub ok 2024-09-19T07:28:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:39 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:39 smithi203 ceph-mon[35335]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:39 smithi203 ceph-mon[35335]: osd.8 [v2:172.21.15.203:6816/4062926645,v1:172.21.15.203:6817/4062926645] boot 2024-09-19T07:28:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:39 smithi203 ceph-mon[35335]: osdmap e61: 9 total, 9 up, 9 in 2024-09-19T07:28:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:39 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:39 smithi189 ceph-mon[34851]: purged_snaps scrub starts 2024-09-19T07:28:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:39 smithi189 ceph-mon[34851]: purged_snaps scrub ok 2024-09-19T07:28:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:39 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:39 smithi189 ceph-mon[34851]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:28:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:39 smithi189 ceph-mon[34851]: osd.8 [v2:172.21.15.203:6816/4062926645,v1:172.21.15.203:6817/4062926645] boot 2024-09-19T07:28:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:39 smithi189 ceph-mon[34851]: osdmap e61: 9 total, 9 up, 9 in 2024-09-19T07:28:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:39 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:28:41.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:40 smithi203 ceph-mon[35335]: osdmap e62: 9 total, 9 up, 9 in 2024-09-19T07:28:41.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:40 smithi203 ceph-mon[35335]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:28:41.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:40 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:28:41.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:40 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:28:41.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:40 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:41.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:40 smithi189 ceph-mon[34851]: osdmap e62: 9 total, 9 up, 9 in 2024-09-19T07:28:41.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:40 smithi189 ceph-mon[34851]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:28:41.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:40 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:28:41.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:40 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:28:41.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:40 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:41.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:40 smithi159 ceph-mon[30472]: osdmap e62: 9 total, 9 up, 9 in 2024-09-19T07:28:41.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:40 smithi159 ceph-mon[30472]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:28:41.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:40 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:28:41.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:40 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:28:41.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:40 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:41.943 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:41 smithi159 ceph-mon[30472]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:28:41.943 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:41 smithi159 ceph-mon[30472]: osdmap e63: 9 total, 9 up, 9 in 2024-09-19T07:28:42.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:41 smithi203 ceph-mon[35335]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:28:42.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:41 smithi203 ceph-mon[35335]: osdmap e63: 9 total, 9 up, 9 in 2024-09-19T07:28:42.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:41 smithi189 ceph-mon[34851]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:28:42.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:41 smithi189 ceph-mon[34851]: osdmap e63: 9 total, 9 up, 9 in 2024-09-19T07:28:43.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:42 smithi203 ceph-mon[35335]: osdmap e64: 9 total, 9 up, 9 in 2024-09-19T07:28:43.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:42 smithi189 ceph-mon[34851]: osdmap e64: 9 total, 9 up, 9 in 2024-09-19T07:28:43.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:42 smithi159 ceph-mon[30472]: osdmap e64: 9 total, 9 up, 9 in 2024-09-19T07:28:44.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:43 smithi203 ceph-mon[35335]: pgmap v210: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:28:44.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:43 smithi203 ceph-mon[35335]: osdmap e65: 9 total, 9 up, 9 in 2024-09-19T07:28:44.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:43 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1227706867' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df29a10f-1c60-41cb-9c2d-bc9f1cb174bc"}]: dispatch 2024-09-19T07:28:44.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:43 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1227706867' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df29a10f-1c60-41cb-9c2d-bc9f1cb174bc"}]': finished 2024-09-19T07:28:44.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:43 smithi203 ceph-mon[35335]: osdmap e66: 10 total, 9 up, 10 in 2024-09-19T07:28:44.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:43 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:28:44.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:43 smithi189 ceph-mon[34851]: pgmap v210: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:28:44.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:43 smithi189 ceph-mon[34851]: osdmap e65: 9 total, 9 up, 9 in 2024-09-19T07:28:44.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:43 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1227706867' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df29a10f-1c60-41cb-9c2d-bc9f1cb174bc"}]: dispatch 2024-09-19T07:28:44.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:43 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1227706867' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df29a10f-1c60-41cb-9c2d-bc9f1cb174bc"}]': finished 2024-09-19T07:28:44.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:43 smithi189 ceph-mon[34851]: osdmap e66: 10 total, 9 up, 10 in 2024-09-19T07:28:44.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:43 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:28:44.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:43 smithi159 ceph-mon[30472]: pgmap v210: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:28:44.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:43 smithi159 ceph-mon[30472]: osdmap e65: 9 total, 9 up, 9 in 2024-09-19T07:28:44.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:43 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1227706867' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df29a10f-1c60-41cb-9c2d-bc9f1cb174bc"}]: dispatch 2024-09-19T07:28:44.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:43 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1227706867' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df29a10f-1c60-41cb-9c2d-bc9f1cb174bc"}]': finished 2024-09-19T07:28:44.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:43 smithi159 ceph-mon[30472]: osdmap e66: 10 total, 9 up, 10 in 2024-09-19T07:28:44.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:43 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:28:45.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:44 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/916022763' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:28:45.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:44 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/916022763' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:28:45.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:44 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/916022763' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:28:46.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:45 smithi203 ceph-mon[35335]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T07:28:46.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:45 smithi189 ceph-mon[34851]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T07:28:46.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:45 smithi159 ceph-mon[30472]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T07:28:47.958 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:47 smithi159 ceph-mon[30472]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T07:28:48.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:47 smithi203 ceph-mon[35335]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T07:28:48.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:47 smithi189 ceph-mon[34851]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T07:28:49.999 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:49 smithi159 ceph-mon[30472]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-19T07:28:50.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:49 smithi203 ceph-mon[35335]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-19T07:28:50.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:49 smithi189 ceph-mon[34851]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-19T07:28:51.984 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:51 smithi159 ceph-mon[30472]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T07:28:52.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:51 smithi203 ceph-mon[35335]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T07:28:52.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:51 smithi189 ceph-mon[34851]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T07:28:53.884 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:53 smithi159 ceph-mon[30472]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T07:28:53.884 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:53 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T07:28:53.884 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:53 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:54.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:53 smithi203 ceph-mon[35335]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T07:28:54.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:53 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T07:28:54.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:53 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:54.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:53 smithi189 ceph-mon[34851]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T07:28:54.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:53 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T07:28:54.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:53 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:55.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:54 smithi203 ceph-mon[35335]: Deploying daemon osd.9 on smithi159 2024-09-19T07:28:55.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:54 smithi189 ceph-mon[34851]: Deploying daemon osd.9 on smithi159 2024-09-19T07:28:55.081 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:54 smithi159 ceph-mon[30472]: Deploying daemon osd.9 on smithi159 2024-09-19T07:28:55.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:55 smithi159 ceph-mon[30472]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:28:56.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:55 smithi203 ceph-mon[35335]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:28:56.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:55 smithi189 ceph-mon[34851]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:28:58.002 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:57 smithi159 ceph-mon[30472]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:28:58.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:57 smithi203 ceph-mon[35335]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:28:58.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:57 smithi189 ceph-mon[34851]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:28:59.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:28:59 smithi159 ceph-mon[30472]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:00.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:28:59 smithi203 ceph-mon[35335]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:00.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:28:59 smithi189 ceph-mon[34851]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:02.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:01 smithi203 ceph-mon[35335]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:02.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:02.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:02.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:02.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:01 smithi189 ceph-mon[34851]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:02.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:02.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:02.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:02.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:01 smithi159 ceph-mon[30472]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:02.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:02.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:02.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:03.959 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:03 smithi159 ceph-mon[30472]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:04.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:03 smithi203 ceph-mon[35335]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:04.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:03 smithi189 ceph-mon[34851]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:05.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:05.319+0000 7f0150c00640 1 -- 172.21.15.159:0/2446694327 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0154062630 con 0x7f01380a3270 2024-09-19T07:29:05.320 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 9 on host 'smithi159' 2024-09-19T07:29:05.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:05.322+0000 7f015a200640 1 -- 172.21.15.159:0/2446694327 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f01380a3270 msgr2=0x7f01380a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:05.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:05.322+0000 7f015a200640 1 --2- 172.21.15.159:0/2446694327 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f01380a3270 0x7f01380a5730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0144009fc0 tx=0x7f0144005f90 comp rx=0 tx=0).stop 2024-09-19T07:29:05.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:05.322+0000 7f015a200640 1 -- 172.21.15.159:0/2446694327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0154104f60 msgr2=0x7f015419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:05.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:05.322+0000 7f015a200640 1 --2- 172.21.15.159:0/2446694327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0154104f60 0x7f015419a5c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f013c00ebc0 tx=0x7f013c00c6a0 comp rx=0 tx=0).stop 2024-09-19T07:29:05.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:05.322+0000 7f015a200640 1 -- 172.21.15.159:0/2446694327 shutdown_connections 2024-09-19T07:29:05.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:05.322+0000 7f015a200640 1 --2- 172.21.15.159:0/2446694327 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f01380a3270 0x7f01380a5730 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:05.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:05.322+0000 7f015a200640 1 --2- 172.21.15.159:0/2446694327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0154105920 0x7f01541a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:05.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:05.322+0000 7f015a200640 1 --2- 172.21.15.159:0/2446694327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0154104f60 0x7f015419a5c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:05.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:05.322+0000 7f015a200640 1 --2- 172.21.15.159:0/2446694327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0154103e00 0x7f015419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:05.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:05.322+0000 7f015a200640 1 -- 172.21.15.159:0/2446694327 >> 172.21.15.159:0/2446694327 conn(0x7f01540ff5b0 msgr2=0x7f01541082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:05.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:05.323+0000 7f015a200640 1 -- 172.21.15.159:0/2446694327 shutdown_connections 2024-09-19T07:29:05.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:05.323+0000 7f015a200640 1 -- 172.21.15.159:0/2446694327 wait complete. 2024-09-19T07:29:05.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:05 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:05.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:05 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:05.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:05 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:05.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:05 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:05.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:05 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:05.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:05 smithi189 ceph-mon[34851]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:05.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:05 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:05.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:05 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:05.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:05 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:05.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:05 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:05.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:05 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:05.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:05 smithi159 ceph-mon[30472]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:05.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:05 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:05.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:05 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:05.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:05 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:05.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:05 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:05.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:05 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:05.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:05 smithi203 ceph-mon[35335]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:06.252 DEBUG:teuthology.orchestra.run.smithi159:osd.9> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.9.service 2024-09-19T07:29:06.256 INFO:tasks.cephadm:Deploying osd.10 on smithi189 with /dev/vg_nvme/lv_1... 2024-09-19T07:29:06.256 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T07:29:06.512 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:29:06.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:06 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:06.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:06 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:06.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:06 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:06.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:06 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:06.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:06 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:06.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:06 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:06.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:06 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:06.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:06 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:06.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:06 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:06.998 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:29:06 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:29:06.746+0000 7f960381d640 -1 osd.9 0 log_to_monitors true 2024-09-19T07:29:07.555 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:07 smithi189 ceph-mon[34851]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:07.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:07 smithi189 ceph-mon[34851]: from='osd.9 [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T07:29:07.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:07 smithi159 ceph-mon[30472]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:07.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:07 smithi159 ceph-mon[30472]: from='osd.9 [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T07:29:07.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:07 smithi203 ceph-mon[35335]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:07.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:07 smithi203 ceph-mon[35335]: from='osd.9 [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T07:29:08.487 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:08 smithi189 ceph-mon[34851]: from='osd.9 [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T07:29:08.487 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:08 smithi189 ceph-mon[34851]: osdmap e67: 10 total, 9 up, 10 in 2024-09-19T07:29:08.487 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:08 smithi189 ceph-mon[34851]: from='osd.9 [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-19T07:29:08.487 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:08 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:29:08.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:29:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:29:08.331+0000 7f95fe400640 -1 osd.9 0 waiting for initial osdmap 2024-09-19T07:29:08.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:29:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:29:08.344+0000 7f95f9400640 -1 osd.9 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:29:08.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:08 smithi159 ceph-mon[30472]: from='osd.9 [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T07:29:08.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:08 smithi159 ceph-mon[30472]: osdmap e67: 10 total, 9 up, 10 in 2024-09-19T07:29:08.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:08 smithi159 ceph-mon[30472]: from='osd.9 [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-19T07:29:08.638 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:08 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:29:08.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:08 smithi203 ceph-mon[35335]: from='osd.9 [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T07:29:08.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:08 smithi203 ceph-mon[35335]: osdmap e67: 10 total, 9 up, 10 in 2024-09-19T07:29:08.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:08 smithi203 ceph-mon[35335]: from='osd.9 [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-19T07:29:08.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:08 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:29:09.312 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-19T07:29:09.342 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_1 2024-09-19T07:29:09.353 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:09 smithi159 ceph-mon[30472]: from='osd.9 [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-19T07:29:09.353 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:09 smithi159 ceph-mon[30472]: osdmap e68: 10 total, 9 up, 10 in 2024-09-19T07:29:09.353 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:09 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:29:09.354 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:09 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:29:09.354 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:09 smithi159 ceph-mon[30472]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:09.354 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:09 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:29:09.592 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:29:09.712 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:09 smithi189 ceph-mon[34851]: from='osd.9 [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-19T07:29:09.712 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:09 smithi189 ceph-mon[34851]: osdmap e68: 10 total, 9 up, 10 in 2024-09-19T07:29:09.712 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:09 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:29:09.712 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:09 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:29:09.713 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:09 smithi189 ceph-mon[34851]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:09.713 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:09 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:29:09.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:09 smithi203 ceph-mon[35335]: from='osd.9 [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-19T07:29:09.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:09 smithi203 ceph-mon[35335]: osdmap e68: 10 total, 9 up, 10 in 2024-09-19T07:29:09.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:09 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:29:09.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:09 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:29:09.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:09 smithi203 ceph-mon[35335]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:29:09.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:09 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:29:10.096 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.070+0000 7fe5e157c640 1 -- 172.21.15.189:0/3271821396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5dc101c00 msgr2=0x7fe5dc102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:10.097 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.070+0000 7fe5e157c640 1 --2- 172.21.15.189:0/3271821396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5dc101c00 0x7fe5dc102000 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc009900 tx=0x7fe5cc02a410 comp rx=0 tx=0).stop 2024-09-19T07:29:10.097 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.070+0000 7fe5e157c640 1 -- 172.21.15.189:0/3271821396 shutdown_connections 2024-09-19T07:29:10.097 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.070+0000 7fe5e157c640 1 --2- 172.21.15.189:0/3271821396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5dc1037c0 0x7fe5dc10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:10.097 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.070+0000 7fe5e157c640 1 --2- 172.21.15.189:0/3271821396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5dc102e00 0x7fe5dc103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:10.097 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.070+0000 7fe5e157c640 1 --2- 172.21.15.189:0/3271821396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5dc101c00 0x7fe5dc102000 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:10.097 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.070+0000 7fe5e157c640 1 -- 172.21.15.189:0/3271821396 >> 172.21.15.189:0/3271821396 conn(0x7fe5dc0fd390 msgr2=0x7fe5dc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:10.097 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.071+0000 7fe5e157c640 1 -- 172.21.15.189:0/3271821396 shutdown_connections 2024-09-19T07:29:10.097 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.071+0000 7fe5e157c640 1 -- 172.21.15.189:0/3271821396 wait complete. 2024-09-19T07:29:10.097 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.071+0000 7fe5e157c640 1 Processor -- start 2024-09-19T07:29:10.097 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.071+0000 7fe5e157c640 1 -- start start 2024-09-19T07:29:10.097 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.071+0000 7fe5e157c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5dc101c00 0x7fe5dc197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:10.097 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.071+0000 7fe5e157c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5dc102e00 0x7fe5dc1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:10.098 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.071+0000 7fe5e157c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5dc1037c0 0x7fe5dc19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:10.098 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.071+0000 7fe5e157c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc10d860 con 0x7fe5dc1037c0 2024-09-19T07:29:10.098 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.071+0000 7fe5e157c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc10d6e0 con 0x7fe5dc102e00 2024-09-19T07:29:10.098 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.071+0000 7fe5e157c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc10d9e0 con 0x7fe5dc101c00 2024-09-19T07:29:10.098 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.072+0000 7fe5da7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5dc102e00 0x7fe5dc1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:10.098 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.072+0000 7fe5da7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5dc102e00 0x7fe5dc1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:42896/0 (socket says 172.21.15.189:42896) 2024-09-19T07:29:10.098 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.072+0000 7fe5da7fc640 1 -- 172.21.15.189:0/1621572572 learned_addr learned my addr 172.21.15.189:0/1621572572 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-19T07:29:10.098 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.072+0000 7fe5daffd640 1 --2- 172.21.15.189:0/1621572572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5dc101c00 0x7fe5dc197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:10.098 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.072+0000 7fe5db7fe640 1 --2- 172.21.15.189:0/1621572572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5dc1037c0 0x7fe5dc19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:10.098 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.072+0000 7fe5da7fc640 1 -- 172.21.15.189:0/1621572572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5dc101c00 msgr2=0x7fe5dc197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:10.098 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.072+0000 7fe5da7fc640 1 --2- 172.21.15.189:0/1621572572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5dc101c00 0x7fe5dc197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:10.099 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.072+0000 7fe5da7fc640 1 -- 172.21.15.189:0/1621572572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5dc1037c0 msgr2=0x7fe5dc19f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:10.099 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.072+0000 7fe5da7fc640 1 --2- 172.21.15.189:0/1621572572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5dc1037c0 0x7fe5dc19f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:10.099 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.072+0000 7fe5da7fc640 1 -- 172.21.15.189:0/1621572572 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc19fab0 con 0x7fe5dc102e00 2024-09-19T07:29:10.099 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.072+0000 7fe5db7fe640 1 --2- 172.21.15.189:0/1621572572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5dc1037c0 0x7fe5dc19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:29:10.099 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.072+0000 7fe5da7fc640 1 --2- 172.21.15.189:0/1621572572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5dc102e00 0x7fe5dc1983c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000c920 tx=0x7fe5d000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:10.099 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.072+0000 7fe5c3fff640 1 -- 172.21.15.189:0/1621572572 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0007c70 con 0x7fe5dc102e00 2024-09-19T07:29:10.099 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.072+0000 7fe5e157c640 1 -- 172.21.15.189:0/1621572572 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5dc19fda0 con 0x7fe5dc102e00 2024-09-19T07:29:10.099 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.072+0000 7fe5c3fff640 1 -- 172.21.15.189:0/1621572572 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d0007e10 con 0x7fe5dc102e00 2024-09-19T07:29:10.099 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.072+0000 7fe5c3fff640 1 -- 172.21.15.189:0/1621572572 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0018890 con 0x7fe5dc102e00 2024-09-19T07:29:10.099 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.072+0000 7fe5e157c640 1 -- 172.21.15.189:0/1621572572 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5dc1a0250 con 0x7fe5dc102e00 2024-09-19T07:29:10.099 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.073+0000 7fe5e157c640 1 -- 172.21.15.189:0/1621572572 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5dc109000 con 0x7fe5dc102e00 2024-09-19T07:29:10.099 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.074+0000 7fe5c3fff640 1 -- 172.21.15.189:0/1621572572 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fe5d0018af0 con 0x7fe5dc102e00 2024-09-19T07:29:10.099 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.075+0000 7fe5c3fff640 1 --2- 172.21.15.189:0/1621572572 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe5c40a3270 0x7fe5c40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:10.099 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.075+0000 7fe5c3fff640 1 -- 172.21.15.189:0/1621572572 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fe5d00cf220 con 0x7fe5dc102e00 2024-09-19T07:29:10.100 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.075+0000 7fe5daffd640 1 --2- 172.21.15.189:0/1621572572 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe5c40a3270 0x7fe5c40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:10.100 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.076+0000 7fe5daffd640 1 --2- 172.21.15.189:0/1621572572 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe5c40a3270 0x7fe5c40a5730 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc005ab0 tx=0x7fe5cc000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:10.100 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.079+0000 7fe5c3fff640 1 -- 172.21.15.189:0/1621572572 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe5d0014030 con 0x7fe5dc102e00 2024-09-19T07:29:10.260 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:10.259+0000 7fe5e157c640 1 -- 172.21.15.189:0/1621572572 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe5dc062630 con 0x7fe5c40a3270 2024-09-19T07:29:10.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: purged_snaps scrub starts 2024-09-19T07:29:10.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: purged_snaps scrub ok 2024-09-19T07:29:10.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: osd.9 [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747] boot 2024-09-19T07:29:10.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: osdmap e69: 10 total, 10 up, 10 in 2024-09-19T07:29:10.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:29:10.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:10.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:10.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:10.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:10.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:10.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:10.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:10.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:10.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:10.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:10.389 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:29:10.389 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:29:10.389 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:10 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:10.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: purged_snaps scrub starts 2024-09-19T07:29:10.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: purged_snaps scrub ok 2024-09-19T07:29:10.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: osd.9 [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747] boot 2024-09-19T07:29:10.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: osdmap e69: 10 total, 10 up, 10 in 2024-09-19T07:29:10.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:29:10.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:10.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:10.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:10.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:10.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:10.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:10.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:10.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:10.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:10.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:10.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:29:10.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:29:10.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:10 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:10.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: purged_snaps scrub starts 2024-09-19T07:29:10.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: purged_snaps scrub ok 2024-09-19T07:29:10.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: osd.9 [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747] boot 2024-09-19T07:29:10.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: osdmap e69: 10 total, 10 up, 10 in 2024-09-19T07:29:10.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:29:10.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:10.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:10.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:10.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:10.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:10.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:10.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:10.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:10.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:10.788 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:10.788 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:29:10.788 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:29:10.788 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:10 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:11.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:11 smithi159 ceph-mon[30472]: Detected new or changed devices on smithi159 2024-09-19T07:29:11.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:11 smithi159 ceph-mon[30472]: Adjusting osd_memory_target on smithi159 to 4272M 2024-09-19T07:29:11.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:11 smithi159 ceph-mon[30472]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:11.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:11 smithi159 ceph-mon[30472]: osdmap e70: 10 total, 10 up, 10 in 2024-09-19T07:29:11.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:11 smithi159 ceph-mon[30472]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:11.737 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:11 smithi189 ceph-mon[34851]: Detected new or changed devices on smithi159 2024-09-19T07:29:11.737 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:11 smithi189 ceph-mon[34851]: Adjusting osd_memory_target on smithi159 to 4272M 2024-09-19T07:29:11.738 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:11 smithi189 ceph-mon[34851]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:11.738 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:11 smithi189 ceph-mon[34851]: osdmap e70: 10 total, 10 up, 10 in 2024-09-19T07:29:11.738 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:11 smithi189 ceph-mon[34851]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:11.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:11 smithi203 ceph-mon[35335]: Detected new or changed devices on smithi159 2024-09-19T07:29:11.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:11 smithi203 ceph-mon[35335]: Adjusting osd_memory_target on smithi159 to 4272M 2024-09-19T07:29:11.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:11 smithi203 ceph-mon[35335]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:11.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:11 smithi203 ceph-mon[35335]: osdmap e70: 10 total, 10 up, 10 in 2024-09-19T07:29:11.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:11 smithi203 ceph-mon[35335]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:14.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:13 smithi203 ceph-mon[35335]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:14.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:13 smithi203 ceph-mon[35335]: from='client.? 172.21.15.189:0/1075961922' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccc440b4-4682-4998-bb53-95e24423cd1b"}]: dispatch 2024-09-19T07:29:14.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:13 smithi203 ceph-mon[35335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccc440b4-4682-4998-bb53-95e24423cd1b"}]: dispatch 2024-09-19T07:29:14.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:13 smithi203 ceph-mon[35335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccc440b4-4682-4998-bb53-95e24423cd1b"}]': finished 2024-09-19T07:29:14.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:13 smithi203 ceph-mon[35335]: osdmap e71: 11 total, 10 up, 11 in 2024-09-19T07:29:14.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:13 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:29:14.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:13 smithi203 ceph-mon[35335]: from='client.? 172.21.15.189:0/864038076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:29:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:13 smithi189 ceph-mon[34851]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:13 smithi189 ceph-mon[34851]: from='client.? 172.21.15.189:0/1075961922' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccc440b4-4682-4998-bb53-95e24423cd1b"}]: dispatch 2024-09-19T07:29:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:13 smithi189 ceph-mon[34851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccc440b4-4682-4998-bb53-95e24423cd1b"}]: dispatch 2024-09-19T07:29:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:13 smithi189 ceph-mon[34851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccc440b4-4682-4998-bb53-95e24423cd1b"}]': finished 2024-09-19T07:29:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:13 smithi189 ceph-mon[34851]: osdmap e71: 11 total, 10 up, 11 in 2024-09-19T07:29:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:13 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:29:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:13 smithi189 ceph-mon[34851]: from='client.? 172.21.15.189:0/864038076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:29:14.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:13 smithi159 ceph-mon[30472]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:14.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:13 smithi159 ceph-mon[30472]: from='client.? 172.21.15.189:0/1075961922' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccc440b4-4682-4998-bb53-95e24423cd1b"}]: dispatch 2024-09-19T07:29:14.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:13 smithi159 ceph-mon[30472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccc440b4-4682-4998-bb53-95e24423cd1b"}]: dispatch 2024-09-19T07:29:14.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:13 smithi159 ceph-mon[30472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccc440b4-4682-4998-bb53-95e24423cd1b"}]': finished 2024-09-19T07:29:14.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:13 smithi159 ceph-mon[30472]: osdmap e71: 11 total, 10 up, 11 in 2024-09-19T07:29:14.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:13 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:29:14.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:13 smithi159 ceph-mon[30472]: from='client.? 172.21.15.189:0/864038076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:29:16.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:15 smithi203 ceph-mon[35335]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:15 smithi189 ceph-mon[34851]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:16.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:15 smithi159 ceph-mon[30472]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:18.003 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:17 smithi159 ceph-mon[30472]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:18.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:17 smithi203 ceph-mon[35335]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:18.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:17 smithi189 ceph-mon[34851]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:20.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:19 smithi203 ceph-mon[35335]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:20.047 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:19 smithi189 ceph-mon[34851]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:20.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:19 smithi159 ceph-mon[30472]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:21 smithi203 ceph-mon[35335]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T07:29:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:22.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:21 smithi189 ceph-mon[34851]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:22.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T07:29:22.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:22.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:21 smithi159 ceph-mon[30472]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:22.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T07:29:22.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:22.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:22 smithi159 ceph-mon[30472]: Deploying daemon osd.10 on smithi189 2024-09-19T07:29:22.973 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:22 smithi189 ceph-mon[34851]: Deploying daemon osd.10 on smithi189 2024-09-19T07:29:23.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:22 smithi203 ceph-mon[35335]: Deploying daemon osd.10 on smithi189 2024-09-19T07:29:24.030 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:23 smithi189 ceph-mon[34851]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:24.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:23 smithi203 ceph-mon[35335]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:24.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:23 smithi159 ceph-mon[30472]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:25.959 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:25 smithi189 ceph-mon[34851]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:26.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:25 smithi203 ceph-mon[35335]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:26.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:25 smithi159 ceph-mon[30472]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:28.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:27 smithi203 ceph-mon[35335]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:28.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:28.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:28.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:27 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:28.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:27 smithi189 ceph-mon[34851]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:28.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:28.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:28.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:27 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:27 smithi159 ceph-mon[30472]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:27 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:30.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:29 smithi203 ceph-mon[35335]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:30.050 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:29 smithi189 ceph-mon[34851]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:30.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:29 smithi159 ceph-mon[30472]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:31.545 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:31.544+0000 7fe5c3fff640 1 -- 172.21.15.189:0/1621572572 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe5dc062630 con 0x7fe5c40a3270 2024-09-19T07:29:31.546 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 10 on host 'smithi189' 2024-09-19T07:29:31.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:31.547+0000 7fe5e157c640 1 -- 172.21.15.189:0/1621572572 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe5c40a3270 msgr2=0x7fe5c40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:31.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:31.547+0000 7fe5e157c640 1 --2- 172.21.15.189:0/1621572572 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe5c40a3270 0x7fe5c40a5730 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc005ab0 tx=0x7fe5cc000bc0 comp rx=0 tx=0).stop 2024-09-19T07:29:31.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:31.547+0000 7fe5e157c640 1 -- 172.21.15.189:0/1621572572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5dc102e00 msgr2=0x7fe5dc1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:31.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:31.547+0000 7fe5e157c640 1 --2- 172.21.15.189:0/1621572572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5dc102e00 0x7fe5dc1983c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000c920 tx=0x7fe5d000cde0 comp rx=0 tx=0).stop 2024-09-19T07:29:31.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:31.547+0000 7fe5e157c640 1 -- 172.21.15.189:0/1621572572 shutdown_connections 2024-09-19T07:29:31.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:31.547+0000 7fe5e157c640 1 --2- 172.21.15.189:0/1621572572 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe5c40a3270 0x7fe5c40a5730 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:31.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:31.547+0000 7fe5e157c640 1 --2- 172.21.15.189:0/1621572572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5dc1037c0 0x7fe5dc19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:31.548 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:31.547+0000 7fe5e157c640 1 --2- 172.21.15.189:0/1621572572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5dc102e00 0x7fe5dc1983c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:31.548 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:31.547+0000 7fe5e157c640 1 --2- 172.21.15.189:0/1621572572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5dc101c00 0x7fe5dc197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:31.548 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:31.547+0000 7fe5e157c640 1 -- 172.21.15.189:0/1621572572 >> 172.21.15.189:0/1621572572 conn(0x7fe5dc0fd390 msgr2=0x7fe5dc1061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:31.548 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:31.547+0000 7fe5e157c640 1 -- 172.21.15.189:0/1621572572 shutdown_connections 2024-09-19T07:29:31.548 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-19T07:29:31.547+0000 7fe5e157c640 1 -- 172.21.15.189:0/1621572572 wait complete. 2024-09-19T07:29:32.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:31 smithi203 ceph-mon[35335]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:32.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:32.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:32.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:32.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:32.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:32.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:32.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:32.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:32.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:31 smithi189 ceph-mon[34851]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:32.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:32.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:32.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:32.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:32.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:32.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:32.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:32.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:32.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:31 smithi159 ceph-mon[30472]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:32.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:32.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:32.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:32.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:32.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:32.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:32.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:32.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:32.266 DEBUG:teuthology.orchestra.run.smithi189:osd.10> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.10.service 2024-09-19T07:29:32.268 INFO:tasks.cephadm:Deploying osd.11 on smithi203 with /dev/vg_nvme/lv_1... 2024-09-19T07:29:32.269 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T07:29:32.523 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:29:33.867 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:33 smithi189 ceph-mon[34851]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:33.867 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:33 smithi189 ceph-mon[34851]: from='osd.10 [v2:172.21.15.189:6824/3018377605,v1:172.21.15.189:6825/3018377605]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T07:29:33.867 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:33 smithi189 ceph-mon[34851]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T07:29:33.867 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:29:33 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:29:33.723+0000 7fc0f7d00640 -1 osd.10 0 log_to_monitors true 2024-09-19T07:29:33.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:33 smithi159 ceph-mon[30472]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:33.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:33 smithi159 ceph-mon[30472]: from='osd.10 [v2:172.21.15.189:6824/3018377605,v1:172.21.15.189:6825/3018377605]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T07:29:33.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:33 smithi159 ceph-mon[30472]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T07:29:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:33 smithi203 ceph-mon[35335]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:33 smithi203 ceph-mon[35335]: from='osd.10 [v2:172.21.15.189:6824/3018377605,v1:172.21.15.189:6825/3018377605]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T07:29:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:33 smithi203 ceph-mon[35335]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T07:29:34.971 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:34 smithi189 ceph-mon[34851]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T07:29:34.971 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:34 smithi189 ceph-mon[34851]: osdmap e72: 11 total, 10 up, 11 in 2024-09-19T07:29:34.971 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:34 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:29:34.972 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:34 smithi189 ceph-mon[34851]: from='osd.10 [v2:172.21.15.189:6824/3018377605,v1:172.21.15.189:6825/3018377605]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:29:34.972 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:34 smithi189 ceph-mon[34851]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:29:34.972 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:29:34 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:29:34.770+0000 7fc0f3c83640 -1 osd.10 0 waiting for initial osdmap 2024-09-19T07:29:34.972 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:29:34 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:29:34.782+0000 7fc0efaab640 -1 osd.10 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:29:35.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:34 smithi203 ceph-mon[35335]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T07:29:35.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:34 smithi203 ceph-mon[35335]: osdmap e72: 11 total, 10 up, 11 in 2024-09-19T07:29:35.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:34 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:29:35.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:34 smithi203 ceph-mon[35335]: from='osd.10 [v2:172.21.15.189:6824/3018377605,v1:172.21.15.189:6825/3018377605]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:29:35.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:34 smithi203 ceph-mon[35335]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:29:35.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:34 smithi159 ceph-mon[30472]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T07:29:35.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:34 smithi159 ceph-mon[30472]: osdmap e72: 11 total, 10 up, 11 in 2024-09-19T07:29:35.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:34 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:29:35.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:34 smithi159 ceph-mon[30472]: from='osd.10 [v2:172.21.15.189:6824/3018377605,v1:172.21.15.189:6825/3018377605]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:29:35.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:34 smithi159 ceph-mon[30472]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-19T07:29:35.433 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-09-19T07:29:35.458 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_1 2024-09-19T07:29:35.719 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:29:36.006 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:35 smithi203 ceph-mon[35335]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:36.006 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:35 smithi203 ceph-mon[35335]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-19T07:29:36.006 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:35 smithi203 ceph-mon[35335]: osdmap e73: 11 total, 10 up, 11 in 2024-09-19T07:29:36.006 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:29:36.007 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:29:36.007 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:36.007 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:36.007 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:36.007 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:36.007 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:36.007 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:36.007 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:36.007 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:36.007 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:36.007 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:35 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:36.026 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:35 smithi189 ceph-mon[34851]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:36.026 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:35 smithi189 ceph-mon[34851]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-19T07:29:36.026 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:35 smithi189 ceph-mon[34851]: osdmap e73: 11 total, 10 up, 11 in 2024-09-19T07:29:36.026 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:29:36.026 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:29:36.026 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:36.026 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:36.026 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:36.026 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:36.026 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:36.026 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:36.027 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:36.027 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:36.027 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:36.027 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:35 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:36.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:35 smithi159 ceph-mon[30472]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:29:36.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:35 smithi159 ceph-mon[30472]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-19T07:29:36.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:35 smithi159 ceph-mon[30472]: osdmap e73: 11 total, 10 up, 11 in 2024-09-19T07:29:36.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:29:36.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:29:36.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:36.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:36.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:36.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:36.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:36.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:36.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:36.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:36.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:36.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:35 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:36.293 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.277+0000 7fce8c9c7640 1 -- 172.21.15.203:0/3556305624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce88101c00 msgr2=0x7fce88102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:36.293 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.277+0000 7fce8c9c7640 1 --2- 172.21.15.203:0/3556305624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce88101c00 0x7fce88102000 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fce78009900 tx=0x7fce7802a410 comp rx=0 tx=0).stop 2024-09-19T07:29:36.293 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.277+0000 7fce8c9c7640 1 -- 172.21.15.203:0/3556305624 shutdown_connections 2024-09-19T07:29:36.294 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.277+0000 7fce8c9c7640 1 --2- 172.21.15.203:0/3556305624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce881037c0 0x7fce8810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:36.294 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.277+0000 7fce8c9c7640 1 --2- 172.21.15.203:0/3556305624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce88102e00 0x7fce88103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:36.294 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.277+0000 7fce8c9c7640 1 --2- 172.21.15.203:0/3556305624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce88101c00 0x7fce88102000 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:36.294 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.277+0000 7fce8c9c7640 1 -- 172.21.15.203:0/3556305624 >> 172.21.15.203:0/3556305624 conn(0x7fce880fd390 msgr2=0x7fce880ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:36.294 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.278+0000 7fce8c9c7640 1 -- 172.21.15.203:0/3556305624 shutdown_connections 2024-09-19T07:29:36.294 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.278+0000 7fce8c9c7640 1 -- 172.21.15.203:0/3556305624 wait complete. 2024-09-19T07:29:36.294 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.278+0000 7fce8c9c7640 1 Processor -- start 2024-09-19T07:29:36.294 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.278+0000 7fce8c9c7640 1 -- start start 2024-09-19T07:29:36.294 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.278+0000 7fce8c9c7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce88101c00 0x7fce881980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:36.294 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.278+0000 7fce8c9c7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce88102e00 0x7fce881985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:36.294 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.278+0000 7fce8c9c7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce881037c0 0x7fce8819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:36.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce8c9c7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fce8810d960 con 0x7fce881037c0 2024-09-19T07:29:36.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce8c9c7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fce8810d7e0 con 0x7fce88102e00 2024-09-19T07:29:36.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce8c9c7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fce8810dae0 con 0x7fce88101c00 2024-09-19T07:29:36.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce8659c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce88101c00 0x7fce881980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:36.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce8659c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce88101c00 0x7fce881980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:41012/0 (socket says 172.21.15.203:41012) 2024-09-19T07:29:36.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce8659c640 1 -- 172.21.15.203:0/3932903699 learned_addr learned my addr 172.21.15.203:0/3932903699 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-19T07:29:36.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce85d9b640 1 --2- 172.21.15.203:0/3932903699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce88102e00 0x7fce881985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:36.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce8659c640 1 -- 172.21.15.203:0/3932903699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce88102e00 msgr2=0x7fce881985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:36.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce8659c640 1 --2- 172.21.15.203:0/3932903699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce88102e00 0x7fce881985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:36.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce8659c640 1 -- 172.21.15.203:0/3932903699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce881037c0 msgr2=0x7fce8819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:36.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce8659c640 1 --2- 172.21.15.203:0/3932903699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce881037c0 0x7fce8819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:36.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce8659c640 1 -- 172.21.15.203:0/3932903699 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8819fcd0 con 0x7fce88101c00 2024-09-19T07:29:36.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce85d9b640 1 --2- 172.21.15.203:0/3932903699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce88102e00 0x7fce881985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:29:36.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce8659c640 1 --2- 172.21.15.203:0/3932903699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce88101c00 0x7fce881980a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fce78031610 tx=0x7fce780043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:36.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce777fe640 1 -- 172.21.15.203:0/3932903699 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7803f030 con 0x7fce88101c00 2024-09-19T07:29:36.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce777fe640 1 -- 172.21.15.203:0/3932903699 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce78003ce0 con 0x7fce88101c00 2024-09-19T07:29:36.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce8c9c7640 1 -- 172.21.15.203:0/3932903699 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8819ff60 con 0x7fce88101c00 2024-09-19T07:29:36.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce777fe640 1 -- 172.21.15.203:0/3932903699 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78035070 con 0x7fce88101c00 2024-09-19T07:29:36.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.279+0000 7fce8c9c7640 1 -- 172.21.15.203:0/3932903699 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce881a0440 con 0x7fce88101c00 2024-09-19T07:29:36.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.280+0000 7fce8c9c7640 1 -- 172.21.15.203:0/3932903699 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce881090f0 con 0x7fce88101c00 2024-09-19T07:29:36.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.281+0000 7fce777fe640 1 -- 172.21.15.203:0/3932903699 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fce78042030 con 0x7fce88101c00 2024-09-19T07:29:36.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.282+0000 7fce777fe640 1 --2- 172.21.15.203:0/3932903699 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fce700a32f0 0x7fce700a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:36.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.283+0000 7fce777fe640 1 -- 172.21.15.203:0/3932903699 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6727+0+0 (secure 0 0 0) 0x7fce780efa60 con 0x7fce88101c00 2024-09-19T07:29:36.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.283+0000 7fce85d9b640 1 --2- 172.21.15.203:0/3932903699 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fce700a32f0 0x7fce700a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:36.296 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.283+0000 7fce85d9b640 1 --2- 172.21.15.203:0/3932903699 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fce700a32f0 0x7fce700a57b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fce881995c0 tx=0x7fce7c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:36.297 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.286+0000 7fce777fe640 1 -- 172.21.15.203:0/3932903699 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fce780bcd10 con 0x7fce88101c00 2024-09-19T07:29:36.468 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:36.468+0000 7fce8c9c7640 1 -- 172.21.15.203:0/3932903699 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fce88062630 con 0x7fce700a32f0 2024-09-19T07:29:37.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:36 smithi203 ceph-mon[35335]: purged_snaps scrub starts 2024-09-19T07:29:37.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:36 smithi203 ceph-mon[35335]: purged_snaps scrub ok 2024-09-19T07:29:37.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:36 smithi203 ceph-mon[35335]: Detected new or changed devices on smithi189 2024-09-19T07:29:37.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:36 smithi203 ceph-mon[35335]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-19T07:29:37.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:36 smithi203 ceph-mon[35335]: osd.10 [v2:172.21.15.189:6824/3018377605,v1:172.21.15.189:6825/3018377605] boot 2024-09-19T07:29:37.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:36 smithi203 ceph-mon[35335]: osdmap e74: 11 total, 11 up, 11 in 2024-09-19T07:29:37.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:36 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:29:37.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:36 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:29:37.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:36 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:29:37.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:36 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:37.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:36 smithi189 ceph-mon[34851]: purged_snaps scrub starts 2024-09-19T07:29:37.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:36 smithi189 ceph-mon[34851]: purged_snaps scrub ok 2024-09-19T07:29:37.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:36 smithi189 ceph-mon[34851]: Detected new or changed devices on smithi189 2024-09-19T07:29:37.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:36 smithi189 ceph-mon[34851]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-19T07:29:37.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:36 smithi189 ceph-mon[34851]: osd.10 [v2:172.21.15.189:6824/3018377605,v1:172.21.15.189:6825/3018377605] boot 2024-09-19T07:29:37.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:36 smithi189 ceph-mon[34851]: osdmap e74: 11 total, 11 up, 11 in 2024-09-19T07:29:37.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:36 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:29:37.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:36 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:29:37.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:36 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:29:37.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:36 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:37.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:36 smithi159 ceph-mon[30472]: purged_snaps scrub starts 2024-09-19T07:29:37.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:36 smithi159 ceph-mon[30472]: purged_snaps scrub ok 2024-09-19T07:29:37.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:36 smithi159 ceph-mon[30472]: Detected new or changed devices on smithi189 2024-09-19T07:29:37.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:36 smithi159 ceph-mon[30472]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-19T07:29:37.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:36 smithi159 ceph-mon[30472]: osd.10 [v2:172.21.15.189:6824/3018377605,v1:172.21.15.189:6825/3018377605] boot 2024-09-19T07:29:37.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:36 smithi159 ceph-mon[30472]: osdmap e74: 11 total, 11 up, 11 in 2024-09-19T07:29:37.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:36 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:29:37.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:36 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:29:37.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:36 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:29:37.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:36 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:38.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:37 smithi203 ceph-mon[35335]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:38.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:37 smithi203 ceph-mon[35335]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:38.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:37 smithi203 ceph-mon[35335]: osdmap e75: 11 total, 11 up, 11 in 2024-09-19T07:29:38.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:37 smithi189 ceph-mon[34851]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:38.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:37 smithi189 ceph-mon[34851]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:38.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:37 smithi189 ceph-mon[34851]: osdmap e75: 11 total, 11 up, 11 in 2024-09-19T07:29:38.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:37 smithi159 ceph-mon[30472]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:38.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:37 smithi159 ceph-mon[30472]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:38.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:37 smithi159 ceph-mon[30472]: osdmap e75: 11 total, 11 up, 11 in 2024-09-19T07:29:39.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:38 smithi203 ceph-mon[35335]: osdmap e76: 11 total, 11 up, 11 in 2024-09-19T07:29:39.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:38 smithi189 ceph-mon[34851]: osdmap e76: 11 total, 11 up, 11 in 2024-09-19T07:29:39.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:38 smithi159 ceph-mon[30472]: osdmap e76: 11 total, 11 up, 11 in 2024-09-19T07:29:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:39 smithi203 ceph-mon[35335]: pgmap v250: 1 pgs: 1 remapped+peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:39 smithi203 ceph-mon[35335]: osdmap e77: 11 total, 11 up, 11 in 2024-09-19T07:29:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:39 smithi203 ceph-mon[35335]: from='client.? 172.21.15.203:0/1155716405' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e84afc5f-8051-4341-8fd0-3afb482ab4df"}]: dispatch 2024-09-19T07:29:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:39 smithi203 ceph-mon[35335]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e84afc5f-8051-4341-8fd0-3afb482ab4df"}]: dispatch 2024-09-19T07:29:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:39 smithi203 ceph-mon[35335]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e84afc5f-8051-4341-8fd0-3afb482ab4df"}]': finished 2024-09-19T07:29:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:39 smithi203 ceph-mon[35335]: osdmap e78: 12 total, 11 up, 12 in 2024-09-19T07:29:40.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:39 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:29:40.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:39 smithi203 ceph-mon[35335]: from='client.? 172.21.15.203:0/34190302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:29:40.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:39 smithi189 ceph-mon[34851]: pgmap v250: 1 pgs: 1 remapped+peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:39 smithi189 ceph-mon[34851]: osdmap e77: 11 total, 11 up, 11 in 2024-09-19T07:29:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:39 smithi189 ceph-mon[34851]: from='client.? 172.21.15.203:0/1155716405' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e84afc5f-8051-4341-8fd0-3afb482ab4df"}]: dispatch 2024-09-19T07:29:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:39 smithi189 ceph-mon[34851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e84afc5f-8051-4341-8fd0-3afb482ab4df"}]: dispatch 2024-09-19T07:29:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:39 smithi189 ceph-mon[34851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e84afc5f-8051-4341-8fd0-3afb482ab4df"}]': finished 2024-09-19T07:29:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:39 smithi189 ceph-mon[34851]: osdmap e78: 12 total, 11 up, 12 in 2024-09-19T07:29:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:39 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:29:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:39 smithi189 ceph-mon[34851]: from='client.? 172.21.15.203:0/34190302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:29:40.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:39 smithi159 ceph-mon[30472]: pgmap v250: 1 pgs: 1 remapped+peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:40.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:39 smithi159 ceph-mon[30472]: osdmap e77: 11 total, 11 up, 11 in 2024-09-19T07:29:40.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:39 smithi159 ceph-mon[30472]: from='client.? 172.21.15.203:0/1155716405' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e84afc5f-8051-4341-8fd0-3afb482ab4df"}]: dispatch 2024-09-19T07:29:40.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:39 smithi159 ceph-mon[30472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e84afc5f-8051-4341-8fd0-3afb482ab4df"}]: dispatch 2024-09-19T07:29:40.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:39 smithi159 ceph-mon[30472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e84afc5f-8051-4341-8fd0-3afb482ab4df"}]': finished 2024-09-19T07:29:40.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:39 smithi159 ceph-mon[30472]: osdmap e78: 12 total, 11 up, 12 in 2024-09-19T07:29:40.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:39 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:29:40.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:39 smithi159 ceph-mon[30472]: from='client.? 172.21.15.203:0/34190302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:29:42.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:41 smithi189 ceph-mon[34851]: pgmap v253: 1 pgs: 1 remapped+peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:42.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:41 smithi159 ceph-mon[30472]: pgmap v253: 1 pgs: 1 remapped+peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:42.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:41 smithi203 ceph-mon[35335]: pgmap v253: 1 pgs: 1 remapped+peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:44.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:43 smithi189 ceph-mon[34851]: pgmap v254: 1 pgs: 1 remapped+peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:44.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:43 smithi159 ceph-mon[30472]: pgmap v254: 1 pgs: 1 remapped+peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:44.152 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:43 smithi203 ceph-mon[35335]: pgmap v254: 1 pgs: 1 remapped+peering; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:45.995 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:45 smithi203 ceph-mon[35335]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-19T07:29:46.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:45 smithi189 ceph-mon[34851]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-19T07:29:46.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:45 smithi159 ceph-mon[30472]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-19T07:29:48.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:47 smithi203 ceph-mon[35335]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T07:29:48.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T07:29:48.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:48.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:47 smithi189 ceph-mon[34851]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T07:29:48.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T07:29:48.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:48.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:47 smithi159 ceph-mon[30472]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T07:29:48.142 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T07:29:48.142 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:49.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:48 smithi159 ceph-mon[30472]: Deploying daemon osd.11 on smithi203 2024-09-19T07:29:49.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:48 smithi203 ceph-mon[35335]: Deploying daemon osd.11 on smithi203 2024-09-19T07:29:49.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:48 smithi189 ceph-mon[34851]: Deploying daemon osd.11 on smithi203 2024-09-19T07:29:49.901 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:49 smithi203 ceph-mon[35335]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T07:29:50.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:49 smithi159 ceph-mon[30472]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T07:29:50.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:49 smithi189 ceph-mon[34851]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T07:29:52.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:51 smithi203 ceph-mon[35335]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T07:29:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:51 smithi159 ceph-mon[30472]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T07:29:52.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:51 smithi189 ceph-mon[34851]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-19T07:29:54.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:53 smithi203 ceph-mon[35335]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:29:54.108 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:53 smithi189 ceph-mon[34851]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:29:54.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:53 smithi159 ceph-mon[30472]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:29:55.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:54 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:55.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:54 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:55.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:54 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:55.123 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:54 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:55.123 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:54 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:55.123 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:54 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:55.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:54 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:55.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:54 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:55.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:54 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:56.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:55 smithi159 ceph-mon[30472]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:29:56.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:55 smithi203 ceph-mon[35335]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:29:56.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:55 smithi189 ceph-mon[34851]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:29:57.622 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:57.621+0000 7fce777fe640 1 -- 172.21.15.203:0/3932903699 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fce88062630 con 0x7fce700a32f0 2024-09-19T07:29:57.622 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 11 on host 'smithi203' 2024-09-19T07:29:57.624 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:57.624+0000 7fce8c9c7640 1 -- 172.21.15.203:0/3932903699 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fce700a32f0 msgr2=0x7fce700a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:57.624 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:57.624+0000 7fce8c9c7640 1 --2- 172.21.15.203:0/3932903699 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fce700a32f0 0x7fce700a57b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fce881995c0 tx=0x7fce7c008040 comp rx=0 tx=0).stop 2024-09-19T07:29:57.624 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:57.624+0000 7fce8c9c7640 1 -- 172.21.15.203:0/3932903699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce88101c00 msgr2=0x7fce881980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:57.624 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:57.624+0000 7fce8c9c7640 1 --2- 172.21.15.203:0/3932903699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce88101c00 0x7fce881980a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fce78031610 tx=0x7fce780043b0 comp rx=0 tx=0).stop 2024-09-19T07:29:57.625 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:57.624+0000 7fce8c9c7640 1 -- 172.21.15.203:0/3932903699 shutdown_connections 2024-09-19T07:29:57.625 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:57.624+0000 7fce8c9c7640 1 --2- 172.21.15.203:0/3932903699 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fce700a32f0 0x7fce700a57b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:57.625 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:57.624+0000 7fce8c9c7640 1 --2- 172.21.15.203:0/3932903699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce881037c0 0x7fce8819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:57.625 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:57.624+0000 7fce8c9c7640 1 --2- 172.21.15.203:0/3932903699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce88102e00 0x7fce881985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:57.625 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:57.624+0000 7fce8c9c7640 1 --2- 172.21.15.203:0/3932903699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce88101c00 0x7fce881980a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:57.625 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:57.624+0000 7fce8c9c7640 1 -- 172.21.15.203:0/3932903699 >> 172.21.15.203:0/3932903699 conn(0x7fce880fd390 msgr2=0x7fce881062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:57.625 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:57.624+0000 7fce8c9c7640 1 -- 172.21.15.203:0/3932903699 shutdown_connections 2024-09-19T07:29:57.625 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:29:57.624+0000 7fce8c9c7640 1 -- 172.21.15.203:0/3932903699 wait complete. 2024-09-19T07:29:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:57 smithi203 ceph-mon[35335]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:57 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:57 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:57 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:57 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:57 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:58.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:57 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:58.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:57 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:58.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:57 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:58.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:57 smithi189 ceph-mon[34851]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:58.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:57 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:58.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:57 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:58.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:57 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:58.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:57 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:58.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:57 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:58.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:57 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:58.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:57 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:58.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:57 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:58.348 DEBUG:teuthology.orchestra.run.smithi203:osd.11> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.11.service 2024-09-19T07:29:58.350 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-19T07:29:58.350 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T07:29:58.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:57 smithi159 ceph-mon[30472]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:58.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:58.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:58.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:58.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:58.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:58.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:58.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:58.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:29:58.730 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:29:59.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:58 smithi159 ceph-mon[30472]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:59.234 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:58 smithi203 ceph-mon[35335]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:59.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:58 smithi189 ceph-mon[34851]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:29:59.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.298+0000 7fada5c00640 1 -- 172.21.15.159:0/502751537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fada01037c0 msgr2=0x7fada010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:59.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.298+0000 7fada5c00640 1 --2- 172.21.15.159:0/502751537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fada01037c0 0x7fada010a050 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fad98009be0 tx=0x7fad9802a6c0 comp rx=0 tx=0).stop 2024-09-19T07:29:59.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.298+0000 7fada5c00640 1 -- 172.21.15.159:0/502751537 shutdown_connections 2024-09-19T07:29:59.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.298+0000 7fada5c00640 1 --2- 172.21.15.159:0/502751537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fada01037c0 0x7fada010a050 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:59.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.298+0000 7fada5c00640 1 --2- 172.21.15.159:0/502751537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fada0102e00 0x7fada0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:59.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.298+0000 7fada5c00640 1 --2- 172.21.15.159:0/502751537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada0101c00 0x7fada0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:59.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.298+0000 7fada5c00640 1 -- 172.21.15.159:0/502751537 >> 172.21.15.159:0/502751537 conn(0x7fada00fd390 msgr2=0x7fada00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:59.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.298+0000 7fada5c00640 1 -- 172.21.15.159:0/502751537 shutdown_connections 2024-09-19T07:29:59.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.298+0000 7fada5c00640 1 -- 172.21.15.159:0/502751537 wait complete. 2024-09-19T07:29:59.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.299+0000 7fada5c00640 1 Processor -- start 2024-09-19T07:29:59.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.299+0000 7fada5c00640 1 -- start start 2024-09-19T07:29:59.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fada5c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fada0101c00 0x7fada0075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:59.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fada5c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fada0102e00 0x7fada00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:59.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fada5c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada01037c0 0x7fada0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:59.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fada5c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fada010d860 con 0x7fada0102e00 2024-09-19T07:29:59.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fada5c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fada010d6e0 con 0x7fada01037c0 2024-09-19T07:29:59.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fada5c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fada010d9e0 con 0x7fada0101c00 2024-09-19T07:29:59.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fad9e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fada0102e00 0x7fada00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:59.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fad9ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fada0101c00 0x7fada0075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:59.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fad9ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fada0101c00 0x7fada0075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:37114/0 (socket says 172.21.15.159:37114) 2024-09-19T07:29:59.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fad9ea00640 1 -- 172.21.15.159:0/1117676459 learned_addr learned my addr 172.21.15.159:0/1117676459 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:29:59.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fad9e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fada0102e00 0x7fada00745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51514/0 (socket says 172.21.15.159:51514) 2024-09-19T07:29:59.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fad9f400640 1 --2- 172.21.15.159:0/1117676459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada01037c0 0x7fada0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:59.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fad9ea00640 1 -- 172.21.15.159:0/1117676459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada01037c0 msgr2=0x7fada0074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:59.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fad9ea00640 1 --2- 172.21.15.159:0/1117676459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada01037c0 0x7fada0074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:59.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fad9ea00640 1 -- 172.21.15.159:0/1117676459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fada0102e00 msgr2=0x7fada00745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:59.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fad9ea00640 1 --2- 172.21.15.159:0/1117676459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fada0102e00 0x7fada00745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:59.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fad9ea00640 1 -- 172.21.15.159:0/1117676459 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada00753e0 con 0x7fada0101c00 2024-09-19T07:29:59.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.300+0000 7fad9ea00640 1 --2- 172.21.15.159:0/1117676459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fada0101c00 0x7fada0075f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fad9000d950 tx=0x7fad9000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:59.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.301+0000 7fad97400640 1 -- 172.21.15.159:0/1117676459 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90017020 con 0x7fada0101c00 2024-09-19T07:29:59.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.301+0000 7fad97400640 1 -- 172.21.15.159:0/1117676459 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad90009c30 con 0x7fada0101c00 2024-09-19T07:29:59.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.301+0000 7fad97400640 1 -- 172.21.15.159:0/1117676459 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9000f720 con 0x7fada0101c00 2024-09-19T07:29:59.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.301+0000 7fada5c00640 1 -- 172.21.15.159:0/1117676459 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada01a1fb0 con 0x7fada0101c00 2024-09-19T07:29:59.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.301+0000 7fada5c00640 1 -- 172.21.15.159:0/1117676459 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada01a2460 con 0x7fada0101c00 2024-09-19T07:29:59.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.302+0000 7fada5c00640 1 -- 172.21.15.159:0/1117676459 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada0109050 con 0x7fada0101c00 2024-09-19T07:29:59.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.304+0000 7fad97400640 1 -- 172.21.15.159:0/1117676459 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fad9001a020 con 0x7fada0101c00 2024-09-19T07:29:59.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.305+0000 7fad97400640 1 --2- 172.21.15.159:0/1117676459 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fad840a32c0 0x7fad840a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:59.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.305+0000 7fad97400640 1 -- 172.21.15.159:0/1117676459 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6809+0+0 (secure 0 0 0) 0x7fad900cf2c0 con 0x7fada0101c00 2024-09-19T07:29:59.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.305+0000 7fad9e000640 1 --2- 172.21.15.159:0/1117676459 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fad840a32c0 0x7fad840a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:59.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.306+0000 7fad9e000640 1 --2- 172.21.15.159:0/1117676459 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fad840a32c0 0x7fad840a5780 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0096c0 tx=0x7fad8c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:59.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.309+0000 7fad97400640 1 -- 172.21.15.159:0/1117676459 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fad90021020 con 0x7fada0101c00 2024-09-19T07:29:59.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.486+0000 7fada5c00640 1 -- 172.21.15.159:0/1117676459 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fada0062e90 con 0x7fada0101c00 2024-09-19T07:29:59.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.486+0000 7fad97400640 1 -- 172.21.15.159:0/1117676459 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) ==== 74+0+133 (secure 0 0 0) 0x7fad90009de0 con 0x7fada0101c00 2024-09-19T07:29:59.486 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:29:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.489+0000 7fada5c00640 1 -- 172.21.15.159:0/1117676459 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fad840a32c0 msgr2=0x7fad840a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.489+0000 7fada5c00640 1 --2- 172.21.15.159:0/1117676459 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fad840a32c0 0x7fad840a5780 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0096c0 tx=0x7fad8c009260 comp rx=0 tx=0).stop 2024-09-19T07:29:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.489+0000 7fada5c00640 1 -- 172.21.15.159:0/1117676459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fada0101c00 msgr2=0x7fada0075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.489+0000 7fada5c00640 1 --2- 172.21.15.159:0/1117676459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fada0101c00 0x7fada0075f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fad9000d950 tx=0x7fad9000de10 comp rx=0 tx=0).stop 2024-09-19T07:29:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.489+0000 7fada5c00640 1 -- 172.21.15.159:0/1117676459 shutdown_connections 2024-09-19T07:29:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.489+0000 7fada5c00640 1 --2- 172.21.15.159:0/1117676459 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fad840a32c0 0x7fad840a5780 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.489+0000 7fada5c00640 1 --2- 172.21.15.159:0/1117676459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada01037c0 0x7fada0074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.489+0000 7fada5c00640 1 --2- 172.21.15.159:0/1117676459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fada0102e00 0x7fada00745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.489+0000 7fada5c00640 1 --2- 172.21.15.159:0/1117676459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fada0101c00 0x7fada0075f90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.489+0000 7fada5c00640 1 -- 172.21.15.159:0/1117676459 >> 172.21.15.159:0/1117676459 conn(0x7fada00fd390 msgr2=0x7fada0106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.489+0000 7fada5c00640 1 -- 172.21.15.159:0/1117676459 shutdown_connections 2024-09-19T07:29:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:29:59.489+0000 7fada5c00640 1 -- 172.21.15.159:0/1117676459 wait complete. 2024-09-19T07:30:00.114 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:29:59 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1117676459' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:30:00.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:29:59 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1117676459' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:30:00.374 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:29:59 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1117676459' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:30:00.416 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":78,"num_osds":12,"num_up_osds":11,"osd_up_since":1726730975,"num_in_osds":12,"osd_in_since":1726730978,"num_remapped_pgs":0} 2024-09-19T07:30:00.536 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:30:00 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:30:00.249+0000 7f887a755640 -1 osd.11 0 log_to_monitors true 2024-09-19T07:30:01.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:00 smithi203 ceph-mon[35335]: overall HEALTH_OK 2024-09-19T07:30:01.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:00 smithi203 ceph-mon[35335]: from='osd.11 [v2:172.21.15.203:6824/2807503891,v1:172.21.15.203:6825/2807503891]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T07:30:01.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:00 smithi203 ceph-mon[35335]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T07:30:01.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:00 smithi203 ceph-mon[35335]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:30:01.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:00 smithi189 ceph-mon[34851]: overall HEALTH_OK 2024-09-19T07:30:01.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:00 smithi189 ceph-mon[34851]: from='osd.11 [v2:172.21.15.203:6824/2807503891,v1:172.21.15.203:6825/2807503891]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T07:30:01.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:00 smithi189 ceph-mon[34851]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T07:30:01.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:00 smithi189 ceph-mon[34851]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:30:01.381 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:00 smithi159 ceph-mon[30472]: overall HEALTH_OK 2024-09-19T07:30:01.381 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:00 smithi159 ceph-mon[30472]: from='osd.11 [v2:172.21.15.203:6824/2807503891,v1:172.21.15.203:6825/2807503891]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T07:30:01.381 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:00 smithi159 ceph-mon[30472]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T07:30:01.381 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:00 smithi159 ceph-mon[30472]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:30:01.417 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T07:30:01.797 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:01.998 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:01 smithi159 ceph-mon[30472]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T07:30:01.998 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:01 smithi159 ceph-mon[30472]: from='osd.11 [v2:172.21.15.203:6824/2807503891,v1:172.21.15.203:6825/2807503891]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:30:01.998 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:01 smithi159 ceph-mon[30472]: osdmap e79: 12 total, 11 up, 12 in 2024-09-19T07:30:01.998 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:30:01.998 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:01 smithi159 ceph-mon[30472]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:30:01.998 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:01 smithi159 ceph-mon[30472]: Detected new or changed devices on smithi203 2024-09-19T07:30:01.998 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:01.999 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:01.999 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:30:01.999 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:30:01.999 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:30:01.999 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:30:01.999 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:01 smithi159 ceph-mon[30472]: Adjusting osd_memory_target on smithi203 to 4272M 2024-09-19T07:30:01.999 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:01.999 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:01.999 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:01.999 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:02.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:01 smithi203 ceph-mon[35335]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T07:30:02.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:01 smithi203 ceph-mon[35335]: from='osd.11 [v2:172.21.15.203:6824/2807503891,v1:172.21.15.203:6825/2807503891]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:30:02.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:01 smithi203 ceph-mon[35335]: osdmap e79: 12 total, 11 up, 12 in 2024-09-19T07:30:02.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:30:02.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:01 smithi203 ceph-mon[35335]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:30:02.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:01 smithi203 ceph-mon[35335]: Detected new or changed devices on smithi203 2024-09-19T07:30:02.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:02.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:02.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:30:02.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:30:02.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:30:02.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:30:02.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:01 smithi203 ceph-mon[35335]: Adjusting osd_memory_target on smithi203 to 4272M 2024-09-19T07:30:02.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:02.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:02.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:02.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:02.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:01 smithi189 ceph-mon[34851]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T07:30:02.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:01 smithi189 ceph-mon[34851]: from='osd.11 [v2:172.21.15.203:6824/2807503891,v1:172.21.15.203:6825/2807503891]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:30:02.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:01 smithi189 ceph-mon[34851]: osdmap e79: 12 total, 11 up, 12 in 2024-09-19T07:30:02.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:30:02.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:01 smithi189 ceph-mon[34851]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-09-19T07:30:02.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:01 smithi189 ceph-mon[34851]: Detected new or changed devices on smithi203 2024-09-19T07:30:02.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:02.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:02.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:30:02.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:30:02.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:30:02.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:30:02.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:01 smithi189 ceph-mon[34851]: Adjusting osd_memory_target on smithi203 to 4272M 2024-09-19T07:30:02.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:02.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:02.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:02.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:02.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.410+0000 7f4f7ce00640 1 -- 172.21.15.159:0/1981466065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f781024e0 msgr2=0x7f4f781048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:02.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.410+0000 7f4f7ce00640 1 --2- 172.21.15.159:0/1981466065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f781024e0 0x7f4f781048d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4f68009900 tx=0x7f4f6802a410 comp rx=0 tx=0).stop 2024-09-19T07:30:02.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.410+0000 7f4f7ce00640 1 -- 172.21.15.159:0/1981466065 shutdown_connections 2024-09-19T07:30:02.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.410+0000 7f4f7ce00640 1 --2- 172.21.15.159:0/1981466065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f78107740 0x7f4f78109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.410+0000 7f4f7ce00640 1 --2- 172.21.15.159:0/1981466065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f78104e10 0x7f4f78107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.410+0000 7f4f7ce00640 1 --2- 172.21.15.159:0/1981466065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f781024e0 0x7f4f781048d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.410+0000 7f4f7ce00640 1 -- 172.21.15.159:0/1981466065 >> 172.21.15.159:0/1981466065 conn(0x7f4f780fc0e0 msgr2=0x7f4f780fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:02.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.410+0000 7f4f7ce00640 1 -- 172.21.15.159:0/1981466065 shutdown_connections 2024-09-19T07:30:02.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.410+0000 7f4f7ce00640 1 -- 172.21.15.159:0/1981466065 wait complete. 2024-09-19T07:30:02.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.411+0000 7f4f7ce00640 1 Processor -- start 2024-09-19T07:30:02.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.411+0000 7f4f7ce00640 1 -- start start 2024-09-19T07:30:02.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.411+0000 7f4f7ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f781024e0 0x7f4f78199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:02.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.411+0000 7f4f7ce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f78104e10 0x7f4f7819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:02.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.411+0000 7f4f7ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f78107740 0x7f4f781a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:02.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.411+0000 7f4f7ce00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7810d710 con 0x7f4f781024e0 2024-09-19T07:30:02.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.411+0000 7f4f7ce00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7810d590 con 0x7f4f78107740 2024-09-19T07:30:02.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.411+0000 7f4f7ce00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7810d890 con 0x7f4f78104e10 2024-09-19T07:30:02.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.411+0000 7f4f77400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f781024e0 0x7f4f78199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:02.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.412+0000 7f4f77400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f781024e0 0x7f4f78199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42756/0 (socket says 172.21.15.159:42756) 2024-09-19T07:30:02.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.412+0000 7f4f77e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f78107740 0x7f4f781a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:02.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.412+0000 7f4f77400640 1 -- 172.21.15.159:0/4191112860 learned_addr learned my addr 172.21.15.159:0/4191112860 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:30:02.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.412+0000 7f4f76a00640 1 --2- 172.21.15.159:0/4191112860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f78104e10 0x7f4f7819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:02.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.412+0000 7f4f77400640 1 -- 172.21.15.159:0/4191112860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f78104e10 msgr2=0x7f4f7819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:02.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.412+0000 7f4f77400640 1 --2- 172.21.15.159:0/4191112860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f78104e10 0x7f4f7819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.412+0000 7f4f77400640 1 -- 172.21.15.159:0/4191112860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f78107740 msgr2=0x7f4f781a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:02.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.412+0000 7f4f77400640 1 --2- 172.21.15.159:0/4191112860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f78107740 0x7f4f781a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.412+0000 7f4f77400640 1 -- 172.21.15.159:0/4191112860 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f781a1b90 con 0x7f4f781024e0 2024-09-19T07:30:02.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.412+0000 7f4f77e00640 1 --2- 172.21.15.159:0/4191112860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f78107740 0x7f4f781a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:30:02.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.412+0000 7f4f76a00640 1 --2- 172.21.15.159:0/4191112860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f78104e10 0x7f4f7819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:30:02.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.412+0000 7f4f77400640 1 --2- 172.21.15.159:0/4191112860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f781024e0 0x7f4f78199f60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4f68009230 tx=0x7f4f68031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:02.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.413+0000 7f4f6fe00640 1 -- 172.21.15.159:0/4191112860 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6803f030 con 0x7f4f781024e0 2024-09-19T07:30:02.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.413+0000 7f4f6fe00640 1 -- 172.21.15.159:0/4191112860 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f68003ad0 con 0x7f4f781024e0 2024-09-19T07:30:02.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.413+0000 7f4f7ce00640 1 -- 172.21.15.159:0/4191112860 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f781a1e20 con 0x7f4f781024e0 2024-09-19T07:30:02.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.413+0000 7f4f6fe00640 1 -- 172.21.15.159:0/4191112860 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68035070 con 0x7f4f781024e0 2024-09-19T07:30:02.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.413+0000 7f4f7ce00640 1 -- 172.21.15.159:0/4191112860 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f781a2270 con 0x7f4f781024e0 2024-09-19T07:30:02.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.414+0000 7f4f7ce00640 1 -- 172.21.15.159:0/4191112860 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f78109290 con 0x7f4f781024e0 2024-09-19T07:30:02.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.416+0000 7f4f6fe00640 1 -- 172.21.15.159:0/4191112860 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f4f68042030 con 0x7f4f781024e0 2024-09-19T07:30:02.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.417+0000 7f4f6fe00640 1 --2- 172.21.15.159:0/4191112860 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4f5c0a32f0 0x7f4f5c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:02.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.417+0000 7f4f6fe00640 1 -- 172.21.15.159:0/4191112860 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7f4f680efbc0 con 0x7f4f781024e0 2024-09-19T07:30:02.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.417+0000 7f4f76a00640 1 --2- 172.21.15.159:0/4191112860 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4f5c0a32f0 0x7f4f5c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:02.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.418+0000 7f4f76a00640 1 --2- 172.21.15.159:0/4191112860 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4f5c0a32f0 0x7f4f5c0a57b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4f7819b480 tx=0x7f4f64005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:02.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.424+0000 7f4f6fe00640 1 -- 172.21.15.159:0/4191112860 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4f680bc7b0 con 0x7f4f781024e0 2024-09-19T07:30:02.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.608+0000 7f4f7ce00640 1 -- 172.21.15.159:0/4191112860 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4f7802da10 con 0x7f4f781024e0 2024-09-19T07:30:02.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.609+0000 7f4f6fe00640 1 -- 172.21.15.159:0/4191112860 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f4f680c0ab0 con 0x7f4f781024e0 2024-09-19T07:30:02.609 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:30:02.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.611+0000 7f4f7ce00640 1 -- 172.21.15.159:0/4191112860 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4f5c0a32f0 msgr2=0x7f4f5c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:02.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.611+0000 7f4f7ce00640 1 --2- 172.21.15.159:0/4191112860 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4f5c0a32f0 0x7f4f5c0a57b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4f7819b480 tx=0x7f4f64005f90 comp rx=0 tx=0).stop 2024-09-19T07:30:02.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.611+0000 7f4f7ce00640 1 -- 172.21.15.159:0/4191112860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f781024e0 msgr2=0x7f4f78199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:02.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.611+0000 7f4f7ce00640 1 --2- 172.21.15.159:0/4191112860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f781024e0 0x7f4f78199f60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4f68009230 tx=0x7f4f68031b60 comp rx=0 tx=0).stop 2024-09-19T07:30:02.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.611+0000 7f4f7ce00640 1 -- 172.21.15.159:0/4191112860 shutdown_connections 2024-09-19T07:30:02.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.611+0000 7f4f7ce00640 1 --2- 172.21.15.159:0/4191112860 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4f5c0a32f0 0x7f4f5c0a57b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.611+0000 7f4f7ce00640 1 --2- 172.21.15.159:0/4191112860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f78107740 0x7f4f781a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.611+0000 7f4f7ce00640 1 --2- 172.21.15.159:0/4191112860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f78104e10 0x7f4f7819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.612+0000 7f4f7ce00640 1 --2- 172.21.15.159:0/4191112860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f781024e0 0x7f4f78199f60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.612+0000 7f4f7ce00640 1 -- 172.21.15.159:0/4191112860 >> 172.21.15.159:0/4191112860 conn(0x7f4f780fc0e0 msgr2=0x7f4f78101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:02.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.612+0000 7f4f7ce00640 1 -- 172.21.15.159:0/4191112860 shutdown_connections 2024-09-19T07:30:02.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:02.612+0000 7f4f7ce00640 1 -- 172.21.15.159:0/4191112860 wait complete. 2024-09-19T07:30:02.786 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:30:02 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:30:02.325+0000 7f88766d8640 -1 osd.11 0 waiting for initial osdmap 2024-09-19T07:30:02.786 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:30:02 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:30:02.337+0000 7f886e4f8640 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:30:03.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:02 smithi203 ceph-mon[35335]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-19T07:30:03.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:02 smithi203 ceph-mon[35335]: osdmap e80: 12 total, 11 up, 12 in 2024-09-19T07:30:03.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:02 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:30:03.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:02 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:30:03.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:02 smithi203 ceph-mon[35335]: from='osd.11 ' entity='osd.11' 2024-09-19T07:30:03.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:02 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/4191112860' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:30:03.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:02 smithi203 ceph-mon[35335]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:30:03.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:02 smithi189 ceph-mon[34851]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-19T07:30:03.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:02 smithi189 ceph-mon[34851]: osdmap e80: 12 total, 11 up, 12 in 2024-09-19T07:30:03.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:02 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:30:03.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:02 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:30:03.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:02 smithi189 ceph-mon[34851]: from='osd.11 ' entity='osd.11' 2024-09-19T07:30:03.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:02 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/4191112860' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:30:03.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:02 smithi189 ceph-mon[34851]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:30:03.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:02 smithi159 ceph-mon[30472]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-09-19T07:30:03.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:02 smithi159 ceph-mon[30472]: osdmap e80: 12 total, 11 up, 12 in 2024-09-19T07:30:03.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:02 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:30:03.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:02 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:30:03.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:02 smithi159 ceph-mon[30472]: from='osd.11 ' entity='osd.11' 2024-09-19T07:30:03.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:02 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/4191112860' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:30:03.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:02 smithi159 ceph-mon[30472]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:30:03.566 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1726730975,"num_in_osds":12,"osd_in_since":1726730978,"num_remapped_pgs":0} 2024-09-19T07:30:04.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:03 smithi203 ceph-mon[35335]: purged_snaps scrub starts 2024-09-19T07:30:04.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:03 smithi203 ceph-mon[35335]: purged_snaps scrub ok 2024-09-19T07:30:04.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:03 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:30:04.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:03 smithi203 ceph-mon[35335]: osd.11 [v2:172.21.15.203:6824/2807503891,v1:172.21.15.203:6825/2807503891] boot 2024-09-19T07:30:04.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:03 smithi203 ceph-mon[35335]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T07:30:04.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:03 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:30:04.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:03 smithi189 ceph-mon[34851]: purged_snaps scrub starts 2024-09-19T07:30:04.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:03 smithi189 ceph-mon[34851]: purged_snaps scrub ok 2024-09-19T07:30:04.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:03 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:30:04.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:03 smithi189 ceph-mon[34851]: osd.11 [v2:172.21.15.203:6824/2807503891,v1:172.21.15.203:6825/2807503891] boot 2024-09-19T07:30:04.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:03 smithi189 ceph-mon[34851]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T07:30:04.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:03 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:30:04.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:03 smithi159 ceph-mon[30472]: purged_snaps scrub starts 2024-09-19T07:30:04.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:03 smithi159 ceph-mon[30472]: purged_snaps scrub ok 2024-09-19T07:30:04.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:03 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:30:04.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:03 smithi159 ceph-mon[30472]: osd.11 [v2:172.21.15.203:6824/2807503891,v1:172.21.15.203:6825/2807503891] boot 2024-09-19T07:30:04.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:03 smithi159 ceph-mon[30472]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T07:30:04.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:03 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:30:04.568 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T07:30:04.948 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:05.241 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:04 smithi159 ceph-mon[30472]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:05.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:04 smithi203 ceph-mon[35335]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:05.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:04 smithi189 ceph-mon[34851]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:05.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.515+0000 7f8793a00640 1 -- 172.21.15.159:0/3692429140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f878c076220 msgr2=0x7f878c074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.515+0000 7f8793a00640 1 --2- 172.21.15.159:0/3692429140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f878c076220 0x7f878c074680 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8780009900 tx=0x7f878002a3f0 comp rx=0 tx=0).stop 2024-09-19T07:30:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.515+0000 7f8793a00640 1 -- 172.21.15.159:0/3692429140 shutdown_connections 2024-09-19T07:30:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.515+0000 7f8793a00640 1 --2- 172.21.15.159:0/3692429140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f878c109a80 0x7f878c10bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.515+0000 7f8793a00640 1 --2- 172.21.15.159:0/3692429140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f878c074c50 0x7f878c0750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.515+0000 7f8793a00640 1 --2- 172.21.15.159:0/3692429140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f878c076220 0x7f878c074680 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.515+0000 7f8793a00640 1 -- 172.21.15.159:0/3692429140 >> 172.21.15.159:0/3692429140 conn(0x7f878c0fd390 msgr2=0x7f878c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.515+0000 7f8793a00640 1 -- 172.21.15.159:0/3692429140 shutdown_connections 2024-09-19T07:30:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.515+0000 7f8793a00640 1 -- 172.21.15.159:0/3692429140 wait complete. 2024-09-19T07:30:05.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.516+0000 7f8793a00640 1 Processor -- start 2024-09-19T07:30:05.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.516+0000 7f8793a00640 1 -- start start 2024-09-19T07:30:05.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.517+0000 7f8793a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f878c074c50 0x7f878c19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:05.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.517+0000 7f8793a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f878c076220 0x7f878c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:05.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.517+0000 7f8793a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f878c109a80 0x7f878c1a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:05.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.517+0000 7f8793a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f878c10fad0 con 0x7f878c074c50 2024-09-19T07:30:05.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.517+0000 7f8793a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f878c10f950 con 0x7f878c109a80 2024-09-19T07:30:05.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.517+0000 7f8793a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f878c10fc50 con 0x7f878c076220 2024-09-19T07:30:05.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.517+0000 7f8791000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f878c074c50 0x7f878c19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:05.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.517+0000 7f8791000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f878c074c50 0x7f878c19a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42778/0 (socket says 172.21.15.159:42778) 2024-09-19T07:30:05.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.517+0000 7f8791000640 1 -- 172.21.15.159:0/3001220518 learned_addr learned my addr 172.21.15.159:0/3001220518 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:30:05.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.517+0000 7f878be00640 1 --2- 172.21.15.159:0/3001220518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f878c076220 0x7f878c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:05.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.517+0000 7f8791a00640 1 --2- 172.21.15.159:0/3001220518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f878c109a80 0x7f878c1a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:05.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.517+0000 7f8791000640 1 -- 172.21.15.159:0/3001220518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f878c076220 msgr2=0x7f878c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.517+0000 7f8791000640 1 --2- 172.21.15.159:0/3001220518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f878c076220 0x7f878c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.517+0000 7f8791000640 1 -- 172.21.15.159:0/3001220518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f878c109a80 msgr2=0x7f878c1a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.517+0000 7f8791000640 1 --2- 172.21.15.159:0/3001220518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f878c109a80 0x7f878c1a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.518+0000 7f8791000640 1 -- 172.21.15.159:0/3001220518 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878c1a1d00 con 0x7f878c074c50 2024-09-19T07:30:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.518+0000 7f8791000640 1 --2- 172.21.15.159:0/3001220518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f878c074c50 0x7f878c19a0d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8780031440 tx=0x7f8780032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.518+0000 7f8789600640 1 -- 172.21.15.159:0/3001220518 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878003f030 con 0x7f878c074c50 2024-09-19T07:30:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.518+0000 7f8789600640 1 -- 172.21.15.159:0/3001220518 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8780003ce0 con 0x7f878c074c50 2024-09-19T07:30:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.518+0000 7f8789600640 1 -- 172.21.15.159:0/3001220518 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780035070 con 0x7f878c074c50 2024-09-19T07:30:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.518+0000 7f8793a00640 1 -- 172.21.15.159:0/3001220518 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878c1a1f90 con 0x7f878c074c50 2024-09-19T07:30:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.518+0000 7f8793a00640 1 -- 172.21.15.159:0/3001220518 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878c1a2390 con 0x7f878c074c50 2024-09-19T07:30:05.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.519+0000 7f8793a00640 1 -- 172.21.15.159:0/3001220518 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f878c108800 con 0x7f878c074c50 2024-09-19T07:30:05.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.521+0000 7f8789600640 1 -- 172.21.15.159:0/3001220518 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f8780042030 con 0x7f878c074c50 2024-09-19T07:30:05.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.523+0000 7f8789600640 1 --2- 172.21.15.159:0/3001220518 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f87740a32f0 0x7f87740a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:05.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.523+0000 7f8789600640 1 -- 172.21.15.159:0/3001220518 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f87800efdc0 con 0x7f878c074c50 2024-09-19T07:30:05.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.523+0000 7f878be00640 1 --2- 172.21.15.159:0/3001220518 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f87740a32f0 0x7f87740a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:05.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.523+0000 7f878be00640 1 --2- 172.21.15.159:0/3001220518 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f87740a32f0 0x7f87740a57b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f878c19b5f0 tx=0x7f877800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:05.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.525+0000 7f8789600640 1 -- 172.21.15.159:0/3001220518 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f87800bc890 con 0x7f878c074c50 2024-09-19T07:30:05.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.703+0000 7f8793a00640 1 -- 172.21.15.159:0/3001220518 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f878c02da10 con 0x7f878c074c50 2024-09-19T07:30:05.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.703+0000 7f8789600640 1 -- 172.21.15.159:0/3001220518 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f87800c0b90 con 0x7f878c074c50 2024-09-19T07:30:05.703 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:30:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.706+0000 7f8793a00640 1 -- 172.21.15.159:0/3001220518 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f87740a32f0 msgr2=0x7f87740a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.706+0000 7f8793a00640 1 --2- 172.21.15.159:0/3001220518 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f87740a32f0 0x7f87740a57b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f878c19b5f0 tx=0x7f877800b390 comp rx=0 tx=0).stop 2024-09-19T07:30:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.706+0000 7f8793a00640 1 -- 172.21.15.159:0/3001220518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f878c074c50 msgr2=0x7f878c19a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.706+0000 7f8793a00640 1 --2- 172.21.15.159:0/3001220518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f878c074c50 0x7f878c19a0d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8780031440 tx=0x7f8780032e20 comp rx=0 tx=0).stop 2024-09-19T07:30:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.706+0000 7f8793a00640 1 -- 172.21.15.159:0/3001220518 shutdown_connections 2024-09-19T07:30:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.706+0000 7f8793a00640 1 --2- 172.21.15.159:0/3001220518 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f87740a32f0 0x7f87740a57b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.706+0000 7f8793a00640 1 --2- 172.21.15.159:0/3001220518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f878c109a80 0x7f878c1a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:05.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.706+0000 7f8793a00640 1 --2- 172.21.15.159:0/3001220518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f878c076220 0x7f878c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:05.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.706+0000 7f8793a00640 1 --2- 172.21.15.159:0/3001220518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f878c074c50 0x7f878c19a0d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:05.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.706+0000 7f8793a00640 1 -- 172.21.15.159:0/3001220518 >> 172.21.15.159:0/3001220518 conn(0x7f878c0fd390 msgr2=0x7f878c1059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:05.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.706+0000 7f8793a00640 1 -- 172.21.15.159:0/3001220518 shutdown_connections 2024-09-19T07:30:05.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:05.706+0000 7f8793a00640 1 -- 172.21.15.159:0/3001220518 wait complete. 2024-09-19T07:30:06.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:05 smithi203 ceph-mon[35335]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T07:30:06.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:05 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3001220518' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:30:06.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:05 smithi189 ceph-mon[34851]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T07:30:06.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:05 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3001220518' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:30:06.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:05 smithi159 ceph-mon[30472]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T07:30:06.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:05 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3001220518' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:30:06.457 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1726731003,"num_in_osds":12,"osd_in_since":1726730978,"num_remapped_pgs":0} 2024-09-19T07:30:06.458 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:30:06.824 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:07.194 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:06 smithi159 ceph-mon[30472]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:07.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:06 smithi203 ceph-mon[35335]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:07.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:06 smithi189 ceph-mon[34851]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:07.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.438+0000 7fa5b9800640 1 -- 172.21.15.159:0/3034335536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5b4103e30 msgr2=0x7fa5b4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:07.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.438+0000 7fa5b9800640 1 --2- 172.21.15.159:0/3034335536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5b4103e30 0x7fa5b4104210 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009900 tx=0x7fa5a402a410 comp rx=0 tx=0).stop 2024-09-19T07:30:07.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.438+0000 7fa5b9800640 1 -- 172.21.15.159:0/3034335536 shutdown_connections 2024-09-19T07:30:07.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.438+0000 7fa5b9800640 1 --2- 172.21.15.159:0/3034335536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5b4108f60 0x7fa5b4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:07.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.438+0000 7fa5b9800640 1 --2- 172.21.15.159:0/3034335536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5b41047e0 0x7fa5b4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:07.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.438+0000 7fa5b9800640 1 --2- 172.21.15.159:0/3034335536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5b4103e30 0x7fa5b4104210 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.438+0000 7fa5b9800640 1 -- 172.21.15.159:0/3034335536 >> 172.21.15.159:0/3034335536 conn(0x7fa5b40ff8d0 msgr2=0x7fa5b4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.438+0000 7fa5b9800640 1 -- 172.21.15.159:0/3034335536 shutdown_connections 2024-09-19T07:30:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.438+0000 7fa5b9800640 1 -- 172.21.15.159:0/3034335536 wait complete. 2024-09-19T07:30:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.439+0000 7fa5b9800640 1 Processor -- start 2024-09-19T07:30:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.440+0000 7fa5b9800640 1 -- start start 2024-09-19T07:30:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.440+0000 7fa5b9800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5b4103e30 0x7fa5b419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.440+0000 7fa5b9800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5b41047e0 0x7fa5b419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.440+0000 7fa5b9800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5b4108f60 0x7fa5b419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.440+0000 7fa5b9800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b4114010 con 0x7fa5b4103e30 2024-09-19T07:30:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.440+0000 7fa5b9800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b4113e90 con 0x7fa5b4108f60 2024-09-19T07:30:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.440+0000 7fa5b9800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b4114190 con 0x7fa5b41047e0 2024-09-19T07:30:07.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.440+0000 7fa5b3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5b4103e30 0x7fa5b419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:07.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.440+0000 7fa5b3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5b4103e30 0x7fa5b419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42792/0 (socket says 172.21.15.159:42792) 2024-09-19T07:30:07.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.440+0000 7fa5abe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5b41047e0 0x7fa5b419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:07.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.440+0000 7fa5b3e00640 1 -- 172.21.15.159:0/3755185117 learned_addr learned my addr 172.21.15.159:0/3755185117 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:30:07.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.440+0000 7fa5b8e00640 1 --2- 172.21.15.159:0/3755185117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5b4108f60 0x7fa5b419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:07.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.441+0000 7fa5b3e00640 1 -- 172.21.15.159:0/3755185117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5b41047e0 msgr2=0x7fa5b419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:07.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.441+0000 7fa5b3e00640 1 --2- 172.21.15.159:0/3755185117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5b41047e0 0x7fa5b419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:07.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.441+0000 7fa5b3e00640 1 -- 172.21.15.159:0/3755185117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5b4108f60 msgr2=0x7fa5b419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:07.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.441+0000 7fa5b3e00640 1 --2- 172.21.15.159:0/3755185117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5b4108f60 0x7fa5b419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:07.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.441+0000 7fa5b3e00640 1 -- 172.21.15.159:0/3755185117 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b419f450 con 0x7fa5b4103e30 2024-09-19T07:30:07.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.441+0000 7fa5abe00640 1 --2- 172.21.15.159:0/3755185117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5b41047e0 0x7fa5b419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:30:07.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.441+0000 7fa5b3e00640 1 --2- 172.21.15.159:0/3755185117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5b4103e30 0x7fa5b419a4a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4004770 tx=0x7fa5a4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:07.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.441+0000 7fa5b1600640 1 -- 172.21.15.159:0/3755185117 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a403f070 con 0x7fa5b4103e30 2024-09-19T07:30:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.441+0000 7fa5b1600640 1 -- 172.21.15.159:0/3755185117 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a4031ca0 con 0x7fa5b4103e30 2024-09-19T07:30:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.441+0000 7fa5b9800640 1 -- 172.21.15.159:0/3755185117 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b419f6e0 con 0x7fa5b4103e30 2024-09-19T07:30:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.441+0000 7fa5b1600640 1 -- 172.21.15.159:0/3755185117 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4035030 con 0x7fa5b4103e30 2024-09-19T07:30:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.441+0000 7fa5b9800640 1 -- 172.21.15.159:0/3755185117 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b41a7020 con 0x7fa5b4103e30 2024-09-19T07:30:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.443+0000 7fa5b9800640 1 -- 172.21.15.159:0/3755185117 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5b4105310 con 0x7fa5b4103e30 2024-09-19T07:30:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.444+0000 7fa5b1600640 1 -- 172.21.15.159:0/3755185117 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa5a4042070 con 0x7fa5b4103e30 2024-09-19T07:30:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.446+0000 7fa5b1600640 1 --2- 172.21.15.159:0/3755185117 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa5980a3240 0x7fa5980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.447+0000 7fa5b1600640 1 -- 172.21.15.159:0/3755185117 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa5a40efb90 con 0x7fa5b4103e30 2024-09-19T07:30:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.447+0000 7fa5abe00640 1 --2- 172.21.15.159:0/3755185117 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa5980a3240 0x7fa5980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.447+0000 7fa5abe00640 1 --2- 172.21.15.159:0/3755185117 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa5980a3240 0x7fa5980a5700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40ffd60 tx=0x7fa59c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.451+0000 7fa5b1600640 1 -- 172.21.15.159:0/3755185117 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa5a40bc660 con 0x7fa5b4103e30 2024-09-19T07:30:07.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.626+0000 7fa5b9800640 1 -- 172.21.15.159:0/3755185117 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5b410c140 con 0x7fa5b4103e30 2024-09-19T07:30:07.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.627+0000 7fa5b1600640 1 -- 172.21.15.159:0/3755185117 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18550 (secure 0 0 0) 0x7fa5a40c0960 con 0x7fa5b4103e30 2024-09-19T07:30:07.627 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:30:07.628 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":82,"fsid":"c9d18048-7657-11ef-bceb-c7b262605968","created":"2024-09-19T07:22:32.059836+0000","modified":"2024-09-19T07:30:04.930606+0000","last_up_change":"2024-09-19T07:30:03.328697+0000","last_in_change":"2024-09-19T07:29:38.987579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:25:55.202457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"752f5466-fe04-408a-8566-86625947fd3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6803","nonce":2738900830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6805","nonce":2738900830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6809","nonce":2738900830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6807","nonce":2738900830}]},"public_addr":"172.21.15.159:6803/2738900830","cluster_addr":"172.21.15.159:6805/2738900830","heartbeat_back_addr":"172.21.15.159:6809/2738900830","heartbeat_front_addr":"172.21.15.159:6807/2738900830","state":["exists","up"]},{"osd":1,"uuid":"7513adda-ca82-4df4-aa99-3b050b410424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6801","nonce":265696178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6803","nonce":265696178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6807","nonce":265696178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6805","nonce":265696178}]},"public_addr":"172.21.15.189:6801/265696178","cluster_addr":"172.21.15.189:6803/265696178","heartbeat_back_addr":"172.21.15.189:6807/265696178","heartbeat_front_addr":"172.21.15.189:6805/265696178","state":["exists","up"]},{"osd":2,"uuid":"aa03be79-58d8-4eb7-a051-0b46c502b12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6801","nonce":1826565288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6803","nonce":1826565288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6807","nonce":1826565288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6805","nonce":1826565288}]},"public_addr":"172.21.15.203:6801/1826565288","cluster_addr":"172.21.15.203:6803/1826565288","heartbeat_back_addr":"172.21.15.203:6807/1826565288","heartbeat_front_addr":"172.21.15.203:6805/1826565288","state":["exists","up"]},{"osd":3,"uuid":"fcd734f1-f40f-4566-82ab-c4607039f23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6811","nonce":109457380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6813","nonce":109457380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6817","nonce":109457380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6815","nonce":109457380}]},"public_addr":"172.21.15.159:6811/109457380","cluster_addr":"172.21.15.159:6813/109457380","heartbeat_back_addr":"172.21.15.159:6817/109457380","heartbeat_front_addr":"172.21.15.159:6815/109457380","state":["exists","up"]},{"osd":4,"uuid":"bc344c9e-2267-4058-bbe1-9c270327db8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6809","nonce":1036888180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6811","nonce":1036888180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6815","nonce":1036888180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6813","nonce":1036888180}]},"public_addr":"172.21.15.189:6809/1036888180","cluster_addr":"172.21.15.189:6811/1036888180","heartbeat_back_addr":"172.21.15.189:6815/1036888180","heartbeat_front_addr":"172.21.15.189:6813/1036888180","state":["exists","up"]},{"osd":5,"uuid":"86c48303-929d-42f4-bf48-cbb32649f95f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6809","nonce":4212601512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6811","nonce":4212601512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6815","nonce":4212601512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6813","nonce":4212601512}]},"public_addr":"172.21.15.203:6809/4212601512","cluster_addr":"172.21.15.203:6811/4212601512","heartbeat_back_addr":"172.21.15.203:6815/4212601512","heartbeat_front_addr":"172.21.15.203:6813/4212601512","state":["exists","up"]},{"osd":6,"uuid":"f0176ed1-c3a7-4d75-b213-fe88fab4441a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6819","nonce":3817684578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6821","nonce":3817684578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6825","nonce":3817684578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6823","nonce":3817684578}]},"public_addr":"172.21.15.159:6819/3817684578","cluster_addr":"172.21.15.159:6821/3817684578","heartbeat_back_addr":"172.21.15.159:6825/3817684578","heartbeat_front_addr":"172.21.15.159:6823/3817684578","state":["exists","up"]},{"osd":7,"uuid":"921bd59d-9415-4559-8e9f-035d336afd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6817","nonce":3936826456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6819","nonce":3936826456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6823","nonce":3936826456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6821","nonce":3936826456}]},"public_addr":"172.21.15.189:6817/3936826456","cluster_addr":"172.21.15.189:6819/3936826456","heartbeat_back_addr":"172.21.15.189:6823/3936826456","heartbeat_front_addr":"172.21.15.189:6821/3936826456","state":["exists","up"]},{"osd":8,"uuid":"ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6817","nonce":4062926645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6819","nonce":4062926645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6823","nonce":4062926645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6821","nonce":4062926645}]},"public_addr":"172.21.15.203:6817/4062926645","cluster_addr":"172.21.15.203:6819/4062926645","heartbeat_back_addr":"172.21.15.203:6823/4062926645","heartbeat_front_addr":"172.21.15.203:6821/4062926645","state":["exists","up"]},{"osd":9,"uuid":"df29a10f-1c60-41cb-9c2d-bc9f1cb174bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6827","nonce":4185848747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6829","nonce":4185848747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6833","nonce":4185848747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6831","nonce":4185848747}]},"public_addr":"172.21.15.159:6827/4185848747","cluster_addr":"172.21.15.159:6829/4185848747","heartbeat_back_addr":"172.21.15.159:6833/4185848747","heartbeat_front_addr":"172.21.15.159:6831/4185848747","state":["exists","up"]},{"osd":10,"uuid":"ccc440b4-4682-4998-bb53-95e24423cd1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6825","nonce":3018377605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6827","nonce":3018377605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6831","nonce":3018377605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6829","nonce":3018377605}]},"public_addr":"172.21.15.189:6825/3018377605","cluster_addr":"172.21.15.189:6827/3018377605","heartbeat_back_addr":"172.21.15.189:6831/3018377605","heartbeat_front_addr":"172.21.15.189:6829/3018377605","state":["exists","up"]},{"osd":11,"uuid":"e84afc5f-8051-4341-8fd0-3afb482ab4df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6825","nonce":2807503891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6827","nonce":2807503891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6831","nonce":2807503891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6829","nonce":2807503891}]},"public_addr":"172.21.15.203:6825/2807503891","cluster_addr":"172.21.15.203:6827/2807503891","heartbeat_back_addr":"172.21.15.203:6831/2807503891","heartbeat_front_addr":"172.21.15.203:6829/2807503891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:00.093677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:25.477205+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:51.345020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:21.077603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:48.822640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:13.516029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:44.382228+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:10.810700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:36.817155+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:07.728553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:34.747540+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:01.249332+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:0/2383222735":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1561371050":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6800/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:6800/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/2108730247":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/3393992615":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1366863724":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/1628344860":"2024-09-20T07:23:03.089190+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:30:07.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.631+0000 7fa5b9800640 1 -- 172.21.15.159:0/3755185117 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa5980a3240 msgr2=0x7fa5980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:07.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.631+0000 7fa5b9800640 1 --2- 172.21.15.159:0/3755185117 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa5980a3240 0x7fa5980a5700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40ffd60 tx=0x7fa59c005f90 comp rx=0 tx=0).stop 2024-09-19T07:30:07.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.631+0000 7fa5b9800640 1 -- 172.21.15.159:0/3755185117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5b4103e30 msgr2=0x7fa5b419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:07.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.631+0000 7fa5b9800640 1 --2- 172.21.15.159:0/3755185117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5b4103e30 0x7fa5b419a4a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4004770 tx=0x7fa5a4003c10 comp rx=0 tx=0).stop 2024-09-19T07:30:07.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.631+0000 7fa5b9800640 1 -- 172.21.15.159:0/3755185117 shutdown_connections 2024-09-19T07:30:07.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.631+0000 7fa5b9800640 1 --2- 172.21.15.159:0/3755185117 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa5980a3240 0x7fa5980a5700 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:07.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.631+0000 7fa5b9800640 1 --2- 172.21.15.159:0/3755185117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5b4108f60 0x7fa5b419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:07.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.631+0000 7fa5b9800640 1 --2- 172.21.15.159:0/3755185117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5b41047e0 0x7fa5b419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:07.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.631+0000 7fa5b9800640 1 --2- 172.21.15.159:0/3755185117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5b4103e30 0x7fa5b419a4a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:07.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.631+0000 7fa5b9800640 1 -- 172.21.15.159:0/3755185117 >> 172.21.15.159:0/3755185117 conn(0x7fa5b40ff8d0 msgr2=0x7fa5b4100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:07.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.631+0000 7fa5b9800640 1 -- 172.21.15.159:0/3755185117 shutdown_connections 2024-09-19T07:30:07.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:07.631+0000 7fa5b9800640 1 -- 172.21.15.159:0/3755185117 wait complete. 2024-09-19T07:30:08.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:07 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3755185117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:30:08.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:07 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3755185117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:30:08.384 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:07 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3755185117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:30:08.425 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T07:25:55.202457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T07:30:08.425 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T07:30:08.785 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:09.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:08 smithi203 ceph-mon[35335]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:09.289 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:08 smithi159 ceph-mon[30472]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:09.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:08 smithi189 ceph-mon[34851]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.402+0000 7f01e1800640 1 -- 172.21.15.159:0/847474984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01dc109120 msgr2=0x7f01dc110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.402+0000 7f01e1800640 1 --2- 172.21.15.159:0/847474984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01dc109120 0x7f01dc110ca0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009ca0 tx=0x7f01d402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:30:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.402+0000 7f01e1800640 1 -- 172.21.15.159:0/847474984 shutdown_connections 2024-09-19T07:30:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.402+0000 7f01e1800640 1 --2- 172.21.15.159:0/847474984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01dc109120 0x7f01dc110ca0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.402+0000 7f01e1800640 1 --2- 172.21.15.159:0/847474984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01dc1049a0 0x7f01dc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.402+0000 7f01e1800640 1 --2- 172.21.15.159:0/847474984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01dc103ff0 0x7f01dc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.402+0000 7f01e1800640 1 -- 172.21.15.159:0/847474984 >> 172.21.15.159:0/847474984 conn(0x7f01dc0ffa90 msgr2=0x7f01dc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.402+0000 7f01e1800640 1 -- 172.21.15.159:0/847474984 shutdown_connections 2024-09-19T07:30:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.402+0000 7f01e1800640 1 -- 172.21.15.159:0/847474984 wait complete. 2024-09-19T07:30:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.403+0000 7f01e1800640 1 Processor -- start 2024-09-19T07:30:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.403+0000 7f01e1800640 1 -- start start 2024-09-19T07:30:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.404+0000 7f01e1800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01dc103ff0 0x7f01dc19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.404+0000 7f01e1800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01dc1049a0 0x7f01dc19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.404+0000 7f01e1800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01dc109120 0x7f01dc19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.404+0000 7f01e1800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc114180 con 0x7f01dc1049a0 2024-09-19T07:30:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.404+0000 7f01e1800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc114000 con 0x7f01dc109120 2024-09-19T07:30:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.404+0000 7f01e1800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc114300 con 0x7f01dc103ff0 2024-09-19T07:30:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.404+0000 7f01da000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01dc1049a0 0x7f01dc19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.404+0000 7f01da000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01dc1049a0 0x7f01dc19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42810/0 (socket says 172.21.15.159:42810) 2024-09-19T07:30:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.404+0000 7f01da000640 1 -- 172.21.15.159:0/3873885915 learned_addr learned my addr 172.21.15.159:0/3873885915 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:30:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.404+0000 7f01daa00640 1 --2- 172.21.15.159:0/3873885915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01dc103ff0 0x7f01dc19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.404+0000 7f01db400640 1 --2- 172.21.15.159:0/3873885915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01dc109120 0x7f01dc19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.404+0000 7f01da000640 1 -- 172.21.15.159:0/3873885915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01dc103ff0 msgr2=0x7f01dc19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.405+0000 7f01da000640 1 --2- 172.21.15.159:0/3873885915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01dc103ff0 0x7f01dc19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.405+0000 7f01da000640 1 -- 172.21.15.159:0/3873885915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01dc109120 msgr2=0x7f01dc19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.405+0000 7f01da000640 1 --2- 172.21.15.159:0/3873885915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01dc109120 0x7f01dc19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.405+0000 7f01da000640 1 -- 172.21.15.159:0/3873885915 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01dc19f5f0 con 0x7f01dc1049a0 2024-09-19T07:30:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.405+0000 7f01daa00640 1 --2- 172.21.15.159:0/3873885915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01dc103ff0 0x7f01dc19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:30:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.405+0000 7f01db400640 1 --2- 172.21.15.159:0/3873885915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01dc109120 0x7f01dc19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:30:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.405+0000 7f01da000640 1 --2- 172.21.15.159:0/3873885915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01dc1049a0 0x7f01dc19ab80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f01c800e9f0 tx=0x7f01c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.405+0000 7f01d3400640 1 -- 172.21.15.159:0/3873885915 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c800cd80 con 0x7f01dc1049a0 2024-09-19T07:30:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.405+0000 7f01d3400640 1 -- 172.21.15.159:0/3873885915 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01c8010910 con 0x7f01dc1049a0 2024-09-19T07:30:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.405+0000 7f01d3400640 1 -- 172.21.15.159:0/3873885915 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8018990 con 0x7f01dc1049a0 2024-09-19T07:30:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.405+0000 7f01e1800640 1 -- 172.21.15.159:0/3873885915 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01dc19f8e0 con 0x7f01dc1049a0 2024-09-19T07:30:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.405+0000 7f01e1800640 1 -- 172.21.15.159:0/3873885915 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01dc1a71c0 con 0x7f01dc1049a0 2024-09-19T07:30:09.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.406+0000 7f01e1800640 1 -- 172.21.15.159:0/3873885915 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01dc1043d0 con 0x7f01dc1049a0 2024-09-19T07:30:09.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.409+0000 7f01d3400640 1 -- 172.21.15.159:0/3873885915 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f01c8010ac0 con 0x7f01dc1049a0 2024-09-19T07:30:09.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.410+0000 7f01d3400640 1 --2- 172.21.15.159:0/3873885915 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f01c00a3240 0x7f01c00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:09.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.411+0000 7f01d3400640 1 -- 172.21.15.159:0/3873885915 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f01c80cf770 con 0x7f01dc1049a0 2024-09-19T07:30:09.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.411+0000 7f01daa00640 1 --2- 172.21.15.159:0/3873885915 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f01c00a3240 0x7f01c00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:09.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.411+0000 7f01daa00640 1 --2- 172.21.15.159:0/3873885915 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f01c00a3240 0x7f01c00a5700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f01cc005d00 tx=0x7f01cc00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:09.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.415+0000 7f01d3400640 1 -- 172.21.15.159:0/3873885915 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f01c809c240 con 0x7f01dc1049a0 2024-09-19T07:30:09.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.588+0000 7f01e1800640 1 -- 172.21.15.159:0/3873885915 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f01dc1a73c0 con 0x7f01dc1049a0 2024-09-19T07:30:09.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.589+0000 7f01d3400640 1 -- 172.21.15.159:0/3873885915 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f01c80a0540 con 0x7f01dc1049a0 2024-09-19T07:30:09.589 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-09-19T07:30:09.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.591+0000 7f01e1800640 1 -- 172.21.15.159:0/3873885915 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f01c00a3240 msgr2=0x7f01c00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:09.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.591+0000 7f01e1800640 1 --2- 172.21.15.159:0/3873885915 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f01c00a3240 0x7f01c00a5700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f01cc005d00 tx=0x7f01cc00b410 comp rx=0 tx=0).stop 2024-09-19T07:30:09.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.591+0000 7f01e1800640 1 -- 172.21.15.159:0/3873885915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01dc1049a0 msgr2=0x7f01dc19ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:09.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.591+0000 7f01e1800640 1 --2- 172.21.15.159:0/3873885915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01dc1049a0 0x7f01dc19ab80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f01c800e9f0 tx=0x7f01c800eeb0 comp rx=0 tx=0).stop 2024-09-19T07:30:09.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.591+0000 7f01e1800640 1 -- 172.21.15.159:0/3873885915 shutdown_connections 2024-09-19T07:30:09.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.591+0000 7f01e1800640 1 --2- 172.21.15.159:0/3873885915 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f01c00a3240 0x7f01c00a5700 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:09.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.591+0000 7f01e1800640 1 --2- 172.21.15.159:0/3873885915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01dc109120 0x7f01dc19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:09.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.591+0000 7f01e1800640 1 --2- 172.21.15.159:0/3873885915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01dc1049a0 0x7f01dc19ab80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:09.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.591+0000 7f01e1800640 1 --2- 172.21.15.159:0/3873885915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01dc103ff0 0x7f01dc19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:09.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.592+0000 7f01e1800640 1 -- 172.21.15.159:0/3873885915 >> 172.21.15.159:0/3873885915 conn(0x7f01dc0ffa90 msgr2=0x7f01dc100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:09.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.592+0000 7f01e1800640 1 -- 172.21.15.159:0/3873885915 shutdown_connections 2024-09-19T07:30:09.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:09.592+0000 7f01e1800640 1 -- 172.21.15.159:0/3873885915 wait complete. 2024-09-19T07:30:10.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:09 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3873885915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:30:10.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:09 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3873885915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:30:10.317 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:09 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3873885915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:30:10.357 INFO:tasks.cephadm:Adding mds.a on smithi159 2024-09-19T07:30:10.357 INFO:tasks.cephadm:Adding mds.d on smithi159 2024-09-19T07:30:10.357 INFO:tasks.cephadm:Adding mds.g on smithi159 2024-09-19T07:30:10.357 INFO:tasks.cephadm:Adding mds.j on smithi159 2024-09-19T07:30:10.357 INFO:tasks.cephadm:Adding mds.b on smithi189 2024-09-19T07:30:10.357 INFO:tasks.cephadm:Adding mds.e on smithi189 2024-09-19T07:30:10.357 INFO:tasks.cephadm:Adding mds.h on smithi189 2024-09-19T07:30:10.357 INFO:tasks.cephadm:Adding mds.k on smithi189 2024-09-19T07:30:10.357 INFO:tasks.cephadm:Adding mds.c on smithi203 2024-09-19T07:30:10.357 INFO:tasks.cephadm:Adding mds.f on smithi203 2024-09-19T07:30:10.358 INFO:tasks.cephadm:Adding mds.i on smithi203 2024-09-19T07:30:10.358 INFO:tasks.cephadm:Adding mds.l on smithi203 2024-09-19T07:30:10.358 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi159=a;smithi159=d;smithi159=g;smithi159=j;smithi189=b;smithi189=e;smithi189=h;smithi189=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l' 2024-09-19T07:30:10.617 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:30:11.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:10 smithi203 ceph-mon[35335]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:11.176 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.161+0000 7f7440d15640 1 -- 172.21.15.203:0/2984155167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f743c102660 msgr2=0x7f743c104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:11.176 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.161+0000 7f7440d15640 1 --2- 172.21.15.203:0/2984155167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f743c102660 0x7f743c104a50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f742c009900 tx=0x7f742c02a410 comp rx=0 tx=0).stop 2024-09-19T07:30:11.176 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.161+0000 7f7440d15640 1 -- 172.21.15.203:0/2984155167 shutdown_connections 2024-09-19T07:30:11.176 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.161+0000 7f7440d15640 1 --2- 172.21.15.203:0/2984155167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f743c1078c0 0x7f743c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:11.176 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.161+0000 7f7440d15640 1 --2- 172.21.15.203:0/2984155167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f743c104f90 0x7f743c107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:11.176 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.161+0000 7f7440d15640 1 --2- 172.21.15.203:0/2984155167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f743c102660 0x7f743c104a50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:11.176 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.161+0000 7f7440d15640 1 -- 172.21.15.203:0/2984155167 >> 172.21.15.203:0/2984155167 conn(0x7f743c0fc280 msgr2=0x7f743c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:11.176 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.161+0000 7f7440d15640 1 -- 172.21.15.203:0/2984155167 shutdown_connections 2024-09-19T07:30:11.176 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.161+0000 7f7440d15640 1 -- 172.21.15.203:0/2984155167 wait complete. 2024-09-19T07:30:11.177 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.162+0000 7f7440d15640 1 Processor -- start 2024-09-19T07:30:11.177 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.162+0000 7f7440d15640 1 -- start start 2024-09-19T07:30:11.177 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.162+0000 7f7440d15640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f743c102660 0x7f743c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:11.177 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.162+0000 7f7440d15640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f743c104f90 0x7f743c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:11.177 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.162+0000 7f7440d15640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f743c1078c0 0x7f743c19f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:11.177 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.162+0000 7f7440d15640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f743c10d930 con 0x7f743c1078c0 2024-09-19T07:30:11.177 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.162+0000 7f7440d15640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f743c10d7b0 con 0x7f743c102660 2024-09-19T07:30:11.177 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.162+0000 7f7440d15640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f743c10dab0 con 0x7f743c104f90 2024-09-19T07:30:11.177 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.162+0000 7f7439d9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f743c104f90 0x7f743c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:11.177 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.162+0000 7f7439d9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f743c104f90 0x7f743c198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:45492/0 (socket says 172.21.15.203:45492) 2024-09-19T07:30:11.178 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.162+0000 7f7439d9b640 1 -- 172.21.15.203:0/802609819 learned_addr learned my addr 172.21.15.203:0/802609819 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-09-19T07:30:11.178 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.163+0000 7f743ad9d640 1 --2- 172.21.15.203:0/802609819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f743c1078c0 0x7f743c19f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:11.178 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.163+0000 7f743a59c640 1 --2- 172.21.15.203:0/802609819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f743c102660 0x7f743c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:11.178 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.163+0000 7f7439d9b640 1 -- 172.21.15.203:0/802609819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f743c102660 msgr2=0x7f743c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:11.178 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.163+0000 7f7439d9b640 1 --2- 172.21.15.203:0/802609819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f743c102660 0x7f743c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:11.178 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.163+0000 7f7439d9b640 1 -- 172.21.15.203:0/802609819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f743c1078c0 msgr2=0x7f743c19f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:11.178 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.163+0000 7f7439d9b640 1 --2- 172.21.15.203:0/802609819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f743c1078c0 0x7f743c19f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:11.178 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.163+0000 7f7439d9b640 1 -- 172.21.15.203:0/802609819 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743c19fc10 con 0x7f743c104f90 2024-09-19T07:30:11.178 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.163+0000 7f743a59c640 1 --2- 172.21.15.203:0/802609819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f743c102660 0x7f743c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:30:11.178 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.163+0000 7f7439d9b640 1 --2- 172.21.15.203:0/802609819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f743c104f90 0x7f743c198520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f743000b450 tx=0x7f743000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:11.178 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.163+0000 7f742b7fe640 1 -- 172.21.15.203:0/802609819 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430015020 con 0x7f743c104f90 2024-09-19T07:30:11.178 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.163+0000 7f742b7fe640 1 -- 172.21.15.203:0/802609819 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7430011d70 con 0x7f743c104f90 2024-09-19T07:30:11.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.163+0000 7f7440d15640 1 -- 172.21.15.203:0/802609819 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743c19ff00 con 0x7f743c104f90 2024-09-19T07:30:11.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.163+0000 7f742b7fe640 1 -- 172.21.15.203:0/802609819 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743000f340 con 0x7f743c104f90 2024-09-19T07:30:11.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.163+0000 7f7440d15640 1 -- 172.21.15.203:0/802609819 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743c1a0430 con 0x7f743c104f90 2024-09-19T07:30:11.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.164+0000 7f7440d15640 1 -- 172.21.15.203:0/802609819 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f743c0fdd70 con 0x7f743c104f90 2024-09-19T07:30:11.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.166+0000 7f742b7fe640 1 -- 172.21.15.203:0/802609819 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f7430018020 con 0x7f743c104f90 2024-09-19T07:30:11.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.166+0000 7f742b7fe640 1 --2- 172.21.15.203:0/802609819 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f74240a3270 0x7f74240a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:11.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.166+0000 7f742b7fe640 1 -- 172.21.15.203:0/802609819 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f74300cf380 con 0x7f743c104f90 2024-09-19T07:30:11.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.166+0000 7f743a59c640 1 --2- 172.21.15.203:0/802609819 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f74240a3270 0x7f74240a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:11.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.167+0000 7f743a59c640 1 --2- 172.21.15.203:0/802609819 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f74240a3270 0x7f74240a5730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f742c02a930 tx=0x7f742c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:11.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.170+0000 7f742b7fe640 1 -- 172.21.15.203:0/802609819 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f743001f020 con 0x7f743c104f90 2024-09-19T07:30:11.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:10 smithi189 ceph-mon[34851]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:11.358 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.357+0000 7f7440d15640 1 -- 172.21.15.203:0/802609819 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi159=a;smithi159=d;smithi159=g;smithi159=j;smithi189=b;smithi189=e;smithi189=h;smithi189=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}) -- 0x7f743c062730 con 0x7f74240a3270 2024-09-19T07:30:11.368 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.367+0000 7f742b7fe640 1 -- 172.21.15.203:0/802609819 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f743c062730 con 0x7f74240a3270 2024-09-19T07:30:11.368 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mds.all update... 2024-09-19T07:30:11.370 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.369+0000 7f7440d15640 1 -- 172.21.15.203:0/802609819 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f74240a3270 msgr2=0x7f74240a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:11.370 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.369+0000 7f7440d15640 1 --2- 172.21.15.203:0/802609819 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f74240a3270 0x7f74240a5730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f742c02a930 tx=0x7f742c000bc0 comp rx=0 tx=0).stop 2024-09-19T07:30:11.370 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.369+0000 7f7440d15640 1 -- 172.21.15.203:0/802609819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f743c104f90 msgr2=0x7f743c198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:11.370 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.369+0000 7f7440d15640 1 --2- 172.21.15.203:0/802609819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f743c104f90 0x7f743c198520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f743000b450 tx=0x7f743000b910 comp rx=0 tx=0).stop 2024-09-19T07:30:11.370 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.369+0000 7f7440d15640 1 -- 172.21.15.203:0/802609819 shutdown_connections 2024-09-19T07:30:11.371 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.369+0000 7f7440d15640 1 --2- 172.21.15.203:0/802609819 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f74240a3270 0x7f74240a5730 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:11.371 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.369+0000 7f7440d15640 1 --2- 172.21.15.203:0/802609819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f743c1078c0 0x7f743c19f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:11.371 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.369+0000 7f7440d15640 1 --2- 172.21.15.203:0/802609819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f743c104f90 0x7f743c198520 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:11.371 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.369+0000 7f7440d15640 1 --2- 172.21.15.203:0/802609819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f743c102660 0x7f743c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:11.371 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.369+0000 7f7440d15640 1 -- 172.21.15.203:0/802609819 >> 172.21.15.203:0/802609819 conn(0x7f743c0fc280 msgr2=0x7f743c102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:11.371 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.369+0000 7f7440d15640 1 -- 172.21.15.203:0/802609819 shutdown_connections 2024-09-19T07:30:11.371 INFO:teuthology.orchestra.run.smithi203.stderr:2024-09-19T07:30:11.369+0000 7f7440d15640 1 -- 172.21.15.203:0/802609819 wait complete. 2024-09-19T07:30:11.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:10 smithi159 ceph-mon[30472]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:12.049 DEBUG:teuthology.orchestra.run.smithi159:mds.a> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.a.service 2024-09-19T07:30:12.053 DEBUG:teuthology.orchestra.run.smithi159:mds.d> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.d.service 2024-09-19T07:30:12.098 DEBUG:teuthology.orchestra.run.smithi159:mds.g> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.g.service 2024-09-19T07:30:12.144 DEBUG:teuthology.orchestra.run.smithi159:mds.j> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.j.service 2024-09-19T07:30:12.189 DEBUG:teuthology.orchestra.run.smithi189:mds.b> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.b.service 2024-09-19T07:30:12.192 DEBUG:teuthology.orchestra.run.smithi189:mds.e> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.e.service 2024-09-19T07:30:12.235 DEBUG:teuthology.orchestra.run.smithi189:mds.h> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.h.service 2024-09-19T07:30:12.279 DEBUG:teuthology.orchestra.run.smithi189:mds.k> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.k.service 2024-09-19T07:30:12.323 DEBUG:teuthology.orchestra.run.smithi203:mds.c> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.c.service 2024-09-19T07:30:12.325 DEBUG:teuthology.orchestra.run.smithi203:mds.f> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.f.service 2024-09-19T07:30:12.368 DEBUG:teuthology.orchestra.run.smithi203:mds.i> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.i.service 2024-09-19T07:30:12.411 DEBUG:teuthology.orchestra.run.smithi203:mds.l> sudo journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.l.service 2024-09-19T07:30:12.456 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-19T07:30:12.456 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:30:12.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:12 smithi189 ceph-mon[34851]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi159=a;smithi159=d;smithi159=g;smithi159=j;smithi189=b;smithi189=e;smithi189=h;smithi189=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:30:12.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:12 smithi189 ceph-mon[34851]: Saving service mds.all spec with placement smithi159=a;smithi159=d;smithi159=g;smithi159=j;smithi189=b;smithi189=e;smithi189=h;smithi189=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-09-19T07:30:12.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:12.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:12.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:12.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:12.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:12.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:12.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:12.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:12.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:12 smithi159 ceph-mon[30472]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi159=a;smithi159=d;smithi159=g;smithi159=j;smithi189=b;smithi189=e;smithi189=h;smithi189=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:30:12.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:12 smithi159 ceph-mon[30472]: Saving service mds.all spec with placement smithi159=a;smithi159=d;smithi159=g;smithi159=j;smithi189=b;smithi189=e;smithi189=h;smithi189=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-09-19T07:30:12.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:12.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:12.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:12.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:12.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:12.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:12.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:12.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:12.647 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:12 smithi203 ceph-mon[35335]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi159=a;smithi159=d;smithi159=g;smithi159=j;smithi189=b;smithi189=e;smithi189=h;smithi189=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:30:12.647 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:12 smithi203 ceph-mon[35335]: Saving service mds.all spec with placement smithi159=a;smithi159=d;smithi159=g;smithi159=j;smithi189=b;smithi189=e;smithi189=h;smithi189=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-09-19T07:30:12.647 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:12.648 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:12.648 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:12.648 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:12.648 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:12.648 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:12.648 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:12.648 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:12.822 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:13.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.442+0000 7f4ff4800640 1 -- 172.21.15.159:0/3867466607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec1049a0 msgr2=0x7f4fec1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:13.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.442+0000 7f4ff4800640 1 --2- 172.21.15.159:0/3867466607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec1049a0 0x7f4fec1089f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009860 tx=0x7f4fdc02a340 comp rx=0 tx=0).stop 2024-09-19T07:30:13.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.442+0000 7f4ff4800640 1 -- 172.21.15.159:0/3867466607 shutdown_connections 2024-09-19T07:30:13.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.442+0000 7f4ff4800640 1 --2- 172.21.15.159:0/3867466607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fec109120 0x7f4fec110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:13.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.442+0000 7f4ff4800640 1 --2- 172.21.15.159:0/3867466607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec1049a0 0x7f4fec1089f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:13.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.442+0000 7f4ff4800640 1 --2- 172.21.15.159:0/3867466607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fec103ff0 0x7f4fec1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:13.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.442+0000 7f4ff4800640 1 -- 172.21.15.159:0/3867466607 >> 172.21.15.159:0/3867466607 conn(0x7f4fec0ffa90 msgr2=0x7f4fec101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:13.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.442+0000 7f4ff4800640 1 -- 172.21.15.159:0/3867466607 shutdown_connections 2024-09-19T07:30:13.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.442+0000 7f4ff4800640 1 -- 172.21.15.159:0/3867466607 wait complete. 2024-09-19T07:30:13.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.443+0000 7f4ff4800640 1 Processor -- start 2024-09-19T07:30:13.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.443+0000 7f4ff4800640 1 -- start start 2024-09-19T07:30:13.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.444+0000 7f4ff4800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec103ff0 0x7f4fec19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:13.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.444+0000 7f4ff4800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fec1049a0 0x7f4fec19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:13.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.444+0000 7f4ff4800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fec109120 0x7f4fec19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:13.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.444+0000 7f4ff4800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec114120 con 0x7f4fec109120 2024-09-19T07:30:13.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.444+0000 7f4ff4800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec113fa0 con 0x7f4fec1049a0 2024-09-19T07:30:13.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.444+0000 7f4ff4800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1142a0 con 0x7f4fec103ff0 2024-09-19T07:30:13.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.444+0000 7f4ff1e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec103ff0 0x7f4fec19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:13.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.444+0000 7f4ff2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fec109120 0x7f4fec19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:13.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.444+0000 7f4ff1e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec103ff0 0x7f4fec19a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:55156/0 (socket says 172.21.15.159:55156) 2024-09-19T07:30:13.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.444+0000 7f4ff2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fec109120 0x7f4fec19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34452/0 (socket says 172.21.15.159:34452) 2024-09-19T07:30:13.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.444+0000 7f4ff1e00640 1 -- 172.21.15.159:0/1322687519 learned_addr learned my addr 172.21.15.159:0/1322687519 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:30:13.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.444+0000 7f4ff1400640 1 --2- 172.21.15.159:0/1322687519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fec1049a0 0x7f4fec19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:13.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.444+0000 7f4ff1e00640 1 -- 172.21.15.159:0/1322687519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fec1049a0 msgr2=0x7f4fec19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:13.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.444+0000 7f4ff1e00640 1 --2- 172.21.15.159:0/1322687519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fec1049a0 0x7f4fec19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:13.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.444+0000 7f4ff1e00640 1 -- 172.21.15.159:0/1322687519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fec109120 msgr2=0x7f4fec19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:13.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.444+0000 7f4ff1e00640 1 --2- 172.21.15.159:0/1322687519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fec109120 0x7f4fec19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:13.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.444+0000 7f4ff1e00640 1 -- 172.21.15.159:0/1322687519 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fec19f650 con 0x7f4fec103ff0 2024-09-19T07:30:13.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.445+0000 7f4ff1e00640 1 --2- 172.21.15.159:0/1322687519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec103ff0 0x7f4fec19a6a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000e9c0 tx=0x7f4fe000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:13.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.445+0000 7f4fe6a00640 1 -- 172.21.15.159:0/1322687519 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe000cde0 con 0x7f4fec103ff0 2024-09-19T07:30:13.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.445+0000 7f4fe6a00640 1 -- 172.21.15.159:0/1322687519 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe00035f0 con 0x7f4fec103ff0 2024-09-19T07:30:13.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.445+0000 7f4ff4800640 1 -- 172.21.15.159:0/1322687519 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fec19f940 con 0x7f4fec103ff0 2024-09-19T07:30:13.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.445+0000 7f4fe6a00640 1 -- 172.21.15.159:0/1322687519 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0010690 con 0x7f4fec103ff0 2024-09-19T07:30:13.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.445+0000 7f4ff4800640 1 -- 172.21.15.159:0/1322687519 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fec1a7220 con 0x7f4fec103ff0 2024-09-19T07:30:13.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.446+0000 7f4ff4800640 1 -- 172.21.15.159:0/1322687519 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fec1043d0 con 0x7f4fec103ff0 2024-09-19T07:30:13.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.448+0000 7f4fe6a00640 1 -- 172.21.15.159:0/1322687519 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f4fe00037a0 con 0x7f4fec103ff0 2024-09-19T07:30:13.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.449+0000 7f4fe6a00640 1 --2- 172.21.15.159:0/1322687519 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4fd40a32c0 0x7f4fd40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:13.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.449+0000 7f4fe6a00640 1 -- 172.21.15.159:0/1322687519 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4fe00cf790 con 0x7f4fec103ff0 2024-09-19T07:30:13.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.449+0000 7f4ff1400640 1 --2- 172.21.15.159:0/1322687519 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4fd40a32c0 0x7f4fd40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:13.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.450+0000 7f4ff1400640 1 --2- 172.21.15.159:0/1322687519 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4fd40a32c0 0x7f4fd40a5780 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc02a930 tx=0x7f4fdc0056d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:13.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.453+0000 7f4fe6a00640 1 -- 172.21.15.159:0/1322687519 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4fe009c8b0 con 0x7f4fec103ff0 2024-09-19T07:30:13.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:13 smithi203 ceph-mon[35335]: Deploying daemon mds.l on smithi203 2024-09-19T07:30:13.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:13 smithi203 ceph-mon[35335]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:13.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:13 smithi159 ceph-mon[30472]: Deploying daemon mds.l on smithi203 2024-09-19T07:30:13.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:13 smithi159 ceph-mon[30472]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:13.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.661+0000 7f4ff4800640 1 -- 172.21.15.159:0/1322687519 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fec10c360 con 0x7f4fec103ff0 2024-09-19T07:30:13.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.662+0000 7f4fe6a00640 1 -- 172.21.15.159:0/1322687519 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f4fe009c1b0 con 0x7f4fec103ff0 2024-09-19T07:30:13.664 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:30:13.664 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":1,"btime":"2024-09-19T07:22:32:059587+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-19T07:30:13.664 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 1 2024-09-19T07:30:13.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.665+0000 7f4ff4800640 1 -- 172.21.15.159:0/1322687519 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4fd40a32c0 msgr2=0x7f4fd40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:13.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.665+0000 7f4ff4800640 1 --2- 172.21.15.159:0/1322687519 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4fd40a32c0 0x7f4fd40a5780 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc02a930 tx=0x7f4fdc0056d0 comp rx=0 tx=0).stop 2024-09-19T07:30:13.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.665+0000 7f4ff4800640 1 -- 172.21.15.159:0/1322687519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec103ff0 msgr2=0x7f4fec19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:13.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.665+0000 7f4ff4800640 1 --2- 172.21.15.159:0/1322687519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec103ff0 0x7f4fec19a6a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000e9c0 tx=0x7f4fe000ee80 comp rx=0 tx=0).stop 2024-09-19T07:30:13.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.665+0000 7f4ff4800640 1 -- 172.21.15.159:0/1322687519 shutdown_connections 2024-09-19T07:30:13.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.665+0000 7f4ff4800640 1 --2- 172.21.15.159:0/1322687519 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4fd40a32c0 0x7f4fd40a5780 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:13.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.665+0000 7f4ff4800640 1 --2- 172.21.15.159:0/1322687519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fec109120 0x7f4fec19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:13.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.665+0000 7f4ff4800640 1 --2- 172.21.15.159:0/1322687519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fec1049a0 0x7f4fec19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:13.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.665+0000 7f4ff4800640 1 --2- 172.21.15.159:0/1322687519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fec103ff0 0x7f4fec19a6a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:13.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.665+0000 7f4ff4800640 1 -- 172.21.15.159:0/1322687519 >> 172.21.15.159:0/1322687519 conn(0x7f4fec0ffa90 msgr2=0x7f4fec100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:13.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.665+0000 7f4ff4800640 1 -- 172.21.15.159:0/1322687519 shutdown_connections 2024-09-19T07:30:13.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:13.665+0000 7f4ff4800640 1 -- 172.21.15.159:0/1322687519 wait complete. 2024-09-19T07:30:13.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:13 smithi189 ceph-mon[34851]: Deploying daemon mds.l on smithi203 2024-09-19T07:30:13.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:13 smithi189 ceph-mon[34851]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:14.212 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 19 07:30:14 smithi203 systemd[1]: Starting Ceph mds.l for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:30:14.477 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:14 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1322687519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:14.478 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 19 07:30:14 smithi203 podman[64861]: 2024-09-19 07:30:14.115208725 +0000 UTC m=+0.015355497 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:30:14.478 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 19 07:30:14 smithi203 podman[64861]: 2024-09-19 07:30:14.241998724 +0000 UTC m=+0.142145494 container create 8c453e9aff1e540afd209aea06960c818c3a7c37068f95b816e6a55fe551c735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-l, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-19T07:30:14.478 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 19 07:30:14 smithi203 podman[64861]: 2024-09-19 07:30:14.478140804 +0000 UTC m=+0.378287566 container init 8c453e9aff1e540afd209aea06960c818c3a7c37068f95b816e6a55fe551c735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T07:30:14.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:14 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1322687519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:14.786 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 19 07:30:14 smithi203 podman[64861]: 2024-09-19 07:30:14.481006586 +0000 UTC m=+0.381153358 container start 8c453e9aff1e540afd209aea06960c818c3a7c37068f95b816e6a55fe551c735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-l, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-19T07:30:14.786 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 19 07:30:14 smithi203 ceph-mds[64880]: -- 172.21.15.203:0/3412148633 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555cc7647860 con 0x555cc7624800 2024-09-19T07:30:14.786 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 19 07:30:14 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-l[64876]: starting mds.l at 2024-09-19T07:30:14.786 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 19 07:30:14 smithi203 bash[64861]: 8c453e9aff1e540afd209aea06960c818c3a7c37068f95b816e6a55fe551c735 2024-09-19T07:30:14.786 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 19 07:30:14 smithi203 systemd[1]: Started Ceph mds.l for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:30:14.805 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:14 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1322687519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:15.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:15 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.203:6832/1415737112,v1:172.21.15.203:6833/1415737112] up:boot 2024-09-19T07:30:15.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:15 smithi203 ceph-mon[35335]: fsmap 1 up:standby 2024-09-19T07:30:15.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:15 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T07:30:15.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:15 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:15.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:15 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:15.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:15 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:15.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:15 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:15.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:15 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:15.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:15 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:15.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:15 smithi203 ceph-mon[35335]: Deploying daemon mds.k on smithi189 2024-09-19T07:30:15.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:15 smithi203 ceph-mon[35335]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:15.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:15 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.203:6832/1415737112,v1:172.21.15.203:6833/1415737112] up:boot 2024-09-19T07:30:15.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:15 smithi189 ceph-mon[34851]: fsmap 1 up:standby 2024-09-19T07:30:15.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:15 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T07:30:15.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:15 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:15.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:15 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:15.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:15 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:15.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:15 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:15.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:15 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:15.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:15 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:15.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:15 smithi189 ceph-mon[34851]: Deploying daemon mds.k on smithi189 2024-09-19T07:30:15.807 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:15 smithi189 ceph-mon[34851]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:15.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:15 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.203:6832/1415737112,v1:172.21.15.203:6833/1415737112] up:boot 2024-09-19T07:30:15.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:15 smithi159 ceph-mon[30472]: fsmap 1 up:standby 2024-09-19T07:30:15.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:15 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T07:30:15.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:15 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:15.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:15 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:15.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:15 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:15.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:15 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:15.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:15 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:15.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:15 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:15.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:15 smithi159 ceph-mon[30472]: Deploying daemon mds.k on smithi189 2024-09-19T07:30:15.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:15 smithi159 ceph-mon[30472]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:16.442 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:30:16.822 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:16.950 INFO:journalctl@ceph.mds.k.smithi189.stdout:Sep 19 07:30:16 smithi189 systemd[1]: Starting Ceph mds.k for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:30:17.236 INFO:journalctl@ceph.mds.k.smithi189.stdout:Sep 19 07:30:16 smithi189 podman[63760]: 2024-09-19 07:30:16.852258192 +0000 UTC m=+0.015956295 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:30:17.236 INFO:journalctl@ceph.mds.k.smithi189.stdout:Sep 19 07:30:16 smithi189 podman[63760]: 2024-09-19 07:30:16.966974819 +0000 UTC m=+0.130672908 container create d03627f586fa3ee2989b402a8a8d4d7e2fa72966577f92ba6eef63d88d71d427 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-k, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T07:30:17.236 INFO:journalctl@ceph.mds.k.smithi189.stdout:Sep 19 07:30:17 smithi189 podman[63760]: 2024-09-19 07:30:17.23661647 +0000 UTC m=+0.400314560 container init d03627f586fa3ee2989b402a8a8d4d7e2fa72966577f92ba6eef63d88d71d427 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T07:30:17.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.397+0000 7f04e9400640 1 -- 172.21.15.159:0/3911141888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04e41019d0 msgr2=0x7f04e4101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:17.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.397+0000 7f04e9400640 1 --2- 172.21.15.159:0/3911141888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04e41019d0 0x7f04e4101e30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009930 tx=0x7f04d002a420 comp rx=0 tx=0).stop 2024-09-19T07:30:17.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.397+0000 7f04e9400640 1 -- 172.21.15.159:0/3911141888 shutdown_connections 2024-09-19T07:30:17.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.397+0000 7f04e9400640 1 --2- 172.21.15.159:0/3911141888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04e4102370 0x7f04e410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:17.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.397+0000 7f04e9400640 1 --2- 172.21.15.159:0/3911141888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04e41019d0 0x7f04e4101e30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:17.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.397+0000 7f04e9400640 1 --2- 172.21.15.159:0/3911141888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04e41079d0 0x7f04e4107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:17.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.397+0000 7f04e9400640 1 -- 172.21.15.159:0/3911141888 >> 172.21.15.159:0/3911141888 conn(0x7f04e40fd710 msgr2=0x7f04e40ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:17.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.397+0000 7f04e9400640 1 -- 172.21.15.159:0/3911141888 shutdown_connections 2024-09-19T07:30:17.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.397+0000 7f04e9400640 1 -- 172.21.15.159:0/3911141888 wait complete. 2024-09-19T07:30:17.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.398+0000 7f04e9400640 1 Processor -- start 2024-09-19T07:30:17.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.398+0000 7f04e9400640 1 -- start start 2024-09-19T07:30:17.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.399+0000 7f04e9400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04e41019d0 0x7f04e40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:17.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.399+0000 7f04e9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04e4102370 0x7f04e4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:17.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.399+0000 7f04e9400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04e41079d0 0x7f04e4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:17.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.399+0000 7f04e9400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4111e20 con 0x7f04e4102370 2024-09-19T07:30:17.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.399+0000 7f04e9400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4111ca0 con 0x7f04e41019d0 2024-09-19T07:30:17.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.399+0000 7f04e9400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4111fa0 con 0x7f04e41079d0 2024-09-19T07:30:17.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.399+0000 7f04e3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04e41019d0 0x7f04e40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:17.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.399+0000 7f04e3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04e41019d0 0x7f04e40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:43068/0 (socket says 172.21.15.159:43068) 2024-09-19T07:30:17.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.399+0000 7f04e3e00640 1 -- 172.21.15.159:0/832983106 learned_addr learned my addr 172.21.15.159:0/832983106 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:30:17.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.399+0000 7f04e8a00640 1 --2- 172.21.15.159:0/832983106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04e41079d0 0x7f04e4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:17.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.399+0000 7f04e3400640 1 --2- 172.21.15.159:0/832983106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04e4102370 0x7f04e4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:17.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.399+0000 7f04e3e00640 1 -- 172.21.15.159:0/832983106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04e41079d0 msgr2=0x7f04e4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:17.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.399+0000 7f04e3e00640 1 --2- 172.21.15.159:0/832983106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04e41079d0 0x7f04e4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:17.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.400+0000 7f04e3e00640 1 -- 172.21.15.159:0/832983106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04e4102370 msgr2=0x7f04e4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:17.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.400+0000 7f04e3e00640 1 --2- 172.21.15.159:0/832983106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04e4102370 0x7f04e4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:17.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.400+0000 7f04e3e00640 1 -- 172.21.15.159:0/832983106 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e41a6760 con 0x7f04e41019d0 2024-09-19T07:30:17.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.400+0000 7f04e3400640 1 --2- 172.21.15.159:0/832983106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04e4102370 0x7f04e4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:30:17.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.400+0000 7f04e3e00640 1 --2- 172.21.15.159:0/832983106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04e41019d0 0x7f04e40744f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f04d400e9f0 tx=0x7f04d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:17.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.401+0000 7f04e0c00640 1 -- 172.21.15.159:0/832983106 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d400cd80 con 0x7f04e41019d0 2024-09-19T07:30:17.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.401+0000 7f04e0c00640 1 -- 172.21.15.159:0/832983106 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d4010910 con 0x7f04e41019d0 2024-09-19T07:30:17.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.401+0000 7f04e9400640 1 -- 172.21.15.159:0/832983106 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e41a6a50 con 0x7f04e41019d0 2024-09-19T07:30:17.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.401+0000 7f04e0c00640 1 -- 172.21.15.159:0/832983106 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4018990 con 0x7f04e41019d0 2024-09-19T07:30:17.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.401+0000 7f04e9400640 1 -- 172.21.15.159:0/832983106 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e41a6fb0 con 0x7f04e41019d0 2024-09-19T07:30:17.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.402+0000 7f04e9400640 1 -- 172.21.15.159:0/832983106 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e4101e30 con 0x7f04e41019d0 2024-09-19T07:30:17.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.404+0000 7f04e0c00640 1 -- 172.21.15.159:0/832983106 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f04d4018b30 con 0x7f04e41019d0 2024-09-19T07:30:17.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.405+0000 7f04e0c00640 1 --2- 172.21.15.159:0/832983106 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f04c80a32c0 0x7f04c80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:17.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.405+0000 7f04e0c00640 1 -- 172.21.15.159:0/832983106 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f04d40cf7f0 con 0x7f04e41019d0 2024-09-19T07:30:17.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.405+0000 7f04e3400640 1 --2- 172.21.15.159:0/832983106 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f04c80a32c0 0x7f04c80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:17.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.406+0000 7f04e3400640 1 --2- 172.21.15.159:0/832983106 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f04c80a32c0 0x7f04c80a5780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f04d0006010 tx=0x7f04d0005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:17.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.409+0000 7f04e0c00640 1 -- 172.21.15.159:0/832983106 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f04d409c910 con 0x7f04e41019d0 2024-09-19T07:30:17.556 INFO:journalctl@ceph.mds.k.smithi189.stdout:Sep 19 07:30:17 smithi189 podman[63760]: 2024-09-19 07:30:17.239611976 +0000 UTC m=+0.403310067 container start d03627f586fa3ee2989b402a8a8d4d7e2fa72966577f92ba6eef63d88d71d427 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-19T07:30:17.556 INFO:journalctl@ceph.mds.k.smithi189.stdout:Sep 19 07:30:17 smithi189 ceph-mds[63779]: -- 172.21.15.189:0/1599584111 <== mon.2 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56356b3bf860 con 0x56356b39d000 2024-09-19T07:30:17.557 INFO:journalctl@ceph.mds.k.smithi189.stdout:Sep 19 07:30:17 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-k[63775]: starting mds.k at 2024-09-19T07:30:17.557 INFO:journalctl@ceph.mds.k.smithi189.stdout:Sep 19 07:30:17 smithi189 bash[63760]: d03627f586fa3ee2989b402a8a8d4d7e2fa72966577f92ba6eef63d88d71d427 2024-09-19T07:30:17.557 INFO:journalctl@ceph.mds.k.smithi189.stdout:Sep 19 07:30:17 smithi189 systemd[1]: Started Ceph mds.k for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:30:17.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.629+0000 7f04e9400640 1 -- 172.21.15.159:0/832983106 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04e4109f60 con 0x7f04e41019d0 2024-09-19T07:30:17.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.630+0000 7f04e0c00640 1 -- 172.21.15.159:0/832983106 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f04d409c210 con 0x7f04e41019d0 2024-09-19T07:30:17.632 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:30:17.632 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":2,"btime":"2024-09-19T07:30:14:509249+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T07:30:17.632 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 2 2024-09-19T07:30:17.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.633+0000 7f04e9400640 1 -- 172.21.15.159:0/832983106 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f04c80a32c0 msgr2=0x7f04c80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:17.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.633+0000 7f04e9400640 1 --2- 172.21.15.159:0/832983106 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f04c80a32c0 0x7f04c80a5780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f04d0006010 tx=0x7f04d0005a20 comp rx=0 tx=0).stop 2024-09-19T07:30:17.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.633+0000 7f04e9400640 1 -- 172.21.15.159:0/832983106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04e41019d0 msgr2=0x7f04e40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:17.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.633+0000 7f04e9400640 1 --2- 172.21.15.159:0/832983106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04e41019d0 0x7f04e40744f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f04d400e9f0 tx=0x7f04d400eeb0 comp rx=0 tx=0).stop 2024-09-19T07:30:17.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.633+0000 7f04e9400640 1 -- 172.21.15.159:0/832983106 shutdown_connections 2024-09-19T07:30:17.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.633+0000 7f04e9400640 1 --2- 172.21.15.159:0/832983106 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f04c80a32c0 0x7f04c80a5780 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:17.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.633+0000 7f04e9400640 1 --2- 172.21.15.159:0/832983106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04e41079d0 0x7f04e4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:17.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.633+0000 7f04e9400640 1 --2- 172.21.15.159:0/832983106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04e4102370 0x7f04e4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:17.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.633+0000 7f04e9400640 1 --2- 172.21.15.159:0/832983106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04e41019d0 0x7f04e40744f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:17.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.633+0000 7f04e9400640 1 -- 172.21.15.159:0/832983106 >> 172.21.15.159:0/832983106 conn(0x7f04e40fd710 msgr2=0x7f04e40fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:17.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.633+0000 7f04e9400640 1 -- 172.21.15.159:0/832983106 shutdown_connections 2024-09-19T07:30:17.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:17.633+0000 7f04e9400640 1 -- 172.21.15.159:0/832983106 wait complete. 2024-09-19T07:30:17.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:17 smithi159 ceph-mon[30472]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:17.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:17 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:17.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:17 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:17.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:17 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:17.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:17 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:17.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:17 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:17.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:17 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:17.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:17 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/832983106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:18.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:17 smithi203 ceph-mon[35335]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:18.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:17 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:18.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:17 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:18.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:17 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:18.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:17 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:18.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:17 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:18.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:17 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:18.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:17 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/832983106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:18.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:17 smithi189 ceph-mon[34851]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:18.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:17 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:18.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:17 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:18.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:17 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:18.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:17 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:18.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:17 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:18.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:17 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:18.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:17 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/832983106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:19.004 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:18 smithi159 ceph-mon[30472]: Deploying daemon mds.j on smithi159 2024-09-19T07:30:19.004 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:18 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.189:6832/2799019079,v1:172.21.15.189:6833/2799019079] up:boot 2024-09-19T07:30:19.004 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:18 smithi159 ceph-mon[30472]: fsmap 2 up:standby 2024-09-19T07:30:19.004 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:18 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T07:30:19.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:18 smithi203 ceph-mon[35335]: Deploying daemon mds.j on smithi159 2024-09-19T07:30:19.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:18 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.189:6832/2799019079,v1:172.21.15.189:6833/2799019079] up:boot 2024-09-19T07:30:19.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:18 smithi203 ceph-mon[35335]: fsmap 2 up:standby 2024-09-19T07:30:19.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:18 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T07:30:19.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:18 smithi189 ceph-mon[34851]: Deploying daemon mds.j on smithi159 2024-09-19T07:30:19.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:18 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.189:6832/2799019079,v1:172.21.15.189:6833/2799019079] up:boot 2024-09-19T07:30:19.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:18 smithi189 ceph-mon[34851]: fsmap 2 up:standby 2024-09-19T07:30:19.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:18 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T07:30:19.858 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:19 smithi159 ceph-mon[30472]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:20.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:19 smithi203 ceph-mon[35335]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:20.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:19 smithi189 ceph-mon[34851]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:20.423 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:30:20.543 INFO:journalctl@ceph.mds.j.smithi159.stdout:Sep 19 07:30:20 smithi159 systemd[1]: Starting Ceph mds.j for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:30:21.137 INFO:journalctl@ceph.mds.j.smithi159.stdout:Sep 19 07:30:20 smithi159 podman[70125]: 2024-09-19 07:30:20.766607185 +0000 UTC m=+0.030108923 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:30:21.137 INFO:journalctl@ceph.mds.j.smithi159.stdout:Sep 19 07:30:20 smithi159 podman[70125]: 2024-09-19 07:30:20.918233677 +0000 UTC m=+0.181735382 container create 235cea9acf8e8d74acab68424ea21a44529506a364ccf54f6bdc98d896d9b51f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-j, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:30:21.398 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:21.541 INFO:journalctl@ceph.mds.j.smithi159.stdout:Sep 19 07:30:21 smithi159 podman[70125]: 2024-09-19 07:30:21.17315105 +0000 UTC m=+0.436652765 container init 235cea9acf8e8d74acab68424ea21a44529506a364ccf54f6bdc98d896d9b51f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T07:30:21.541 INFO:journalctl@ceph.mds.j.smithi159.stdout:Sep 19 07:30:21 smithi159 podman[70125]: 2024-09-19 07:30:21.179520932 +0000 UTC m=+0.443022640 container start 235cea9acf8e8d74acab68424ea21a44529506a364ccf54f6bdc98d896d9b51f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-j, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-09-19T07:30:21.541 INFO:journalctl@ceph.mds.j.smithi159.stdout:Sep 19 07:30:21 smithi159 ceph-mds[70160]: -- 172.21.15.159:0/1044257851 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d2c48f0680 con 0x55d2c48c5400 2024-09-19T07:30:21.541 INFO:journalctl@ceph.mds.j.smithi159.stdout:Sep 19 07:30:21 smithi159 ceph-mds[70160]: -- 172.21.15.159:0/1044257851 <== mon.0 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d2c48e7860 con 0x55d2c48c5400 2024-09-19T07:30:21.541 INFO:journalctl@ceph.mds.j.smithi159.stdout:Sep 19 07:30:21 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-j[70156]: starting mds.j at 2024-09-19T07:30:21.542 INFO:journalctl@ceph.mds.j.smithi159.stdout:Sep 19 07:30:21 smithi159 bash[70125]: 235cea9acf8e8d74acab68424ea21a44529506a364ccf54f6bdc98d896d9b51f 2024-09-19T07:30:21.542 INFO:journalctl@ceph.mds.j.smithi159.stdout:Sep 19 07:30:21 smithi159 systemd[1]: Started Ceph mds.j for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:30:21.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:21 smithi159 ceph-mon[30472]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:21.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:21.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:21.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:21.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:21.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:21.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:21 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:22.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.007+0000 7fb9c2a00640 1 -- 172.21.15.159:0/4284012964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9bc1049c0 msgr2=0x7fb9bc108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:22.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.007+0000 7fb9c2a00640 1 --2- 172.21.15.159:0/4284012964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9bc1049c0 0x7fb9bc108a10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009900 tx=0x7fb9a402a410 comp rx=0 tx=0).stop 2024-09-19T07:30:22.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.007+0000 7fb9c2a00640 1 -- 172.21.15.159:0/4284012964 shutdown_connections 2024-09-19T07:30:22.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.007+0000 7fb9c2a00640 1 --2- 172.21.15.159:0/4284012964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9bc109140 0x7fb9bc110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:22.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.007+0000 7fb9c2a00640 1 --2- 172.21.15.159:0/4284012964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9bc1049c0 0x7fb9bc108a10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:22.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.007+0000 7fb9c2a00640 1 --2- 172.21.15.159:0/4284012964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9bc104010 0x7fb9bc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:22.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.007+0000 7fb9c2a00640 1 -- 172.21.15.159:0/4284012964 >> 172.21.15.159:0/4284012964 conn(0x7fb9bc0ffa50 msgr2=0x7fb9bc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:22.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.007+0000 7fb9c2a00640 1 -- 172.21.15.159:0/4284012964 shutdown_connections 2024-09-19T07:30:22.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.007+0000 7fb9c2a00640 1 -- 172.21.15.159:0/4284012964 wait complete. 2024-09-19T07:30:22.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.008+0000 7fb9c2a00640 1 Processor -- start 2024-09-19T07:30:22.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.008+0000 7fb9c2a00640 1 -- start start 2024-09-19T07:30:22.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.008+0000 7fb9c2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9bc104010 0x7fb9bc19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:22.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.008+0000 7fb9c2a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9bc1049c0 0x7fb9bc19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:22.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.008+0000 7fb9c2a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9bc109140 0x7fb9bc19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:22.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.008+0000 7fb9c2a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1141b0 con 0x7fb9bc104010 2024-09-19T07:30:22.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.008+0000 7fb9c2a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc114030 con 0x7fb9bc109140 2024-09-19T07:30:22.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.008+0000 7fb9c2a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc114330 con 0x7fb9bc1049c0 2024-09-19T07:30:22.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.008+0000 7fb9bbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9bc104010 0x7fb9bc19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:22.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.008+0000 7fb9bbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9bc104010 0x7fb9bc19a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47098/0 (socket says 172.21.15.159:47098) 2024-09-19T07:30:22.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.008+0000 7fb9bbe00640 1 -- 172.21.15.159:0/1372801990 learned_addr learned my addr 172.21.15.159:0/1372801990 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:30:22.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.008+0000 7fb9b3400640 1 --2- 172.21.15.159:0/1372801990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9bc1049c0 0x7fb9bc19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:22.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.009+0000 7fb9c0a00640 1 --2- 172.21.15.159:0/1372801990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9bc109140 0x7fb9bc19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:22.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.009+0000 7fb9bbe00640 1 -- 172.21.15.159:0/1372801990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9bc1049c0 msgr2=0x7fb9bc19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:22.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.009+0000 7fb9bbe00640 1 --2- 172.21.15.159:0/1372801990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9bc1049c0 0x7fb9bc19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:22.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.009+0000 7fb9bbe00640 1 -- 172.21.15.159:0/1372801990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9bc109140 msgr2=0x7fb9bc19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:22.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.009+0000 7fb9bbe00640 1 --2- 172.21.15.159:0/1372801990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9bc109140 0x7fb9bc19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:22.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.009+0000 7fb9bbe00640 1 -- 172.21.15.159:0/1372801990 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9bc19f6a0 con 0x7fb9bc104010 2024-09-19T07:30:22.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.009+0000 7fb9bbe00640 1 --2- 172.21.15.159:0/1372801990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9bc104010 0x7fb9bc19a6f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00e9f0 tx=0x7fb9ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:22.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.009+0000 7fb9b9600640 1 -- 172.21.15.159:0/1372801990 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac00cd80 con 0x7fb9bc104010 2024-09-19T07:30:22.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.010+0000 7fb9b9600640 1 -- 172.21.15.159:0/1372801990 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9ac010910 con 0x7fb9bc104010 2024-09-19T07:30:22.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.010+0000 7fb9c2a00640 1 -- 172.21.15.159:0/1372801990 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9bc19f990 con 0x7fb9bc104010 2024-09-19T07:30:22.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.010+0000 7fb9b9600640 1 -- 172.21.15.159:0/1372801990 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac018990 con 0x7fb9bc104010 2024-09-19T07:30:22.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.010+0000 7fb9c2a00640 1 -- 172.21.15.159:0/1372801990 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9bc1a7270 con 0x7fb9bc104010 2024-09-19T07:30:22.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.011+0000 7fb9c2a00640 1 -- 172.21.15.159:0/1372801990 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9bc1043f0 con 0x7fb9bc104010 2024-09-19T07:30:22.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.013+0000 7fb9b9600640 1 -- 172.21.15.159:0/1372801990 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb9ac010ac0 con 0x7fb9bc104010 2024-09-19T07:30:22.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.014+0000 7fb9b9600640 1 --2- 172.21.15.159:0/1372801990 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb9a00a3240 0x7fb9a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:22.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.014+0000 7fb9b9600640 1 -- 172.21.15.159:0/1372801990 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb9ac0cf820 con 0x7fb9bc104010 2024-09-19T07:30:22.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.014+0000 7fb9b3400640 1 --2- 172.21.15.159:0/1372801990 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb9a00a3240 0x7fb9a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:22.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.015+0000 7fb9b3400640 1 --2- 172.21.15.159:0/1372801990 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb9a00a3240 0x7fb9a00a5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4000c00 tx=0x7fb9a4005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:22.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.019+0000 7fb9b9600640 1 -- 172.21.15.159:0/1372801990 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb9ac09c2f0 con 0x7fb9bc104010 2024-09-19T07:30:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:21 smithi203 ceph-mon[35335]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:21 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:22.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:21 smithi189 ceph-mon[34851]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:22.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:22.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:22.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:22.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:22.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:22.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:21 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:22.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.240+0000 7fb9c2a00640 1 -- 172.21.15.159:0/1372801990 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9bc19ba50 con 0x7fb9bc104010 2024-09-19T07:30:22.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.241+0000 7fb9b9600640 1 -- 172.21.15.159:0/1372801990 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7fb9ac0a05f0 con 0x7fb9bc104010 2024-09-19T07:30:22.242 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:30:22.242 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":4,"btime":"2024-09-19T07:30:21:751871+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-19T07:30:22.242 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 4 2024-09-19T07:30:22.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.243+0000 7fb9c2a00640 1 -- 172.21.15.159:0/1372801990 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb9a00a3240 msgr2=0x7fb9a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:22.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.243+0000 7fb9c2a00640 1 --2- 172.21.15.159:0/1372801990 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb9a00a3240 0x7fb9a00a5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4000c00 tx=0x7fb9a4005b50 comp rx=0 tx=0).stop 2024-09-19T07:30:22.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.243+0000 7fb9c2a00640 1 -- 172.21.15.159:0/1372801990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9bc104010 msgr2=0x7fb9bc19a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:22.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.243+0000 7fb9c2a00640 1 --2- 172.21.15.159:0/1372801990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9bc104010 0x7fb9bc19a6f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00e9f0 tx=0x7fb9ac00eeb0 comp rx=0 tx=0).stop 2024-09-19T07:30:22.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.243+0000 7fb9c2a00640 1 -- 172.21.15.159:0/1372801990 shutdown_connections 2024-09-19T07:30:22.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.243+0000 7fb9c2a00640 1 --2- 172.21.15.159:0/1372801990 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb9a00a3240 0x7fb9a00a5700 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:22.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.243+0000 7fb9c2a00640 1 --2- 172.21.15.159:0/1372801990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9bc109140 0x7fb9bc19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:22.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.243+0000 7fb9c2a00640 1 --2- 172.21.15.159:0/1372801990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9bc1049c0 0x7fb9bc19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:22.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.243+0000 7fb9c2a00640 1 --2- 172.21.15.159:0/1372801990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9bc104010 0x7fb9bc19a6f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:22.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.243+0000 7fb9c2a00640 1 -- 172.21.15.159:0/1372801990 >> 172.21.15.159:0/1372801990 conn(0x7fb9bc0ffa50 msgr2=0x7fb9bc100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:22.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.243+0000 7fb9c2a00640 1 -- 172.21.15.159:0/1372801990 shutdown_connections 2024-09-19T07:30:22.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:22.243+0000 7fb9c2a00640 1 -- 172.21.15.159:0/1372801990 wait complete. 2024-09-19T07:30:22.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:22 smithi159 ceph-mon[30472]: Deploying daemon mds.d on smithi159 2024-09-19T07:30:22.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:22 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.159:6834/205226977,v1:172.21.15.159:6835/205226977] up:boot 2024-09-19T07:30:22.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:22 smithi159 ceph-mon[30472]: fsmap 3 up:standby 2024-09-19T07:30:22.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:22 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T07:30:22.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:22 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1372801990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:23.038 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:22 smithi203 ceph-mon[35335]: Deploying daemon mds.d on smithi159 2024-09-19T07:30:23.038 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:22 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.159:6834/205226977,v1:172.21.15.159:6835/205226977] up:boot 2024-09-19T07:30:23.038 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:22 smithi203 ceph-mon[35335]: fsmap 3 up:standby 2024-09-19T07:30:23.038 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:22 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T07:30:23.038 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:22 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1372801990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:23.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:22 smithi189 ceph-mon[34851]: Deploying daemon mds.d on smithi159 2024-09-19T07:30:23.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:22 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.159:6834/205226977,v1:172.21.15.159:6835/205226977] up:boot 2024-09-19T07:30:23.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:22 smithi189 ceph-mon[34851]: fsmap 3 up:standby 2024-09-19T07:30:23.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:22 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T07:30:23.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:22 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1372801990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:24.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:23 smithi203 ceph-mon[35335]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:24.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:23 smithi189 ceph-mon[34851]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:24.057 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:23 smithi159 ceph-mon[30472]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:24.730 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:30:24 smithi159 systemd[1]: Starting Ceph mds.d for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:30:25.055 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:30:24 smithi159 podman[70525]: 2024-09-19 07:30:24.634107153 +0000 UTC m=+0.027201902 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:30:25.055 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:30:24 smithi159 podman[70525]: 2024-09-19 07:30:24.777575257 +0000 UTC m=+0.170670015 container create 637c6bb5df7e439b32a6d5072ac29026a1e734af8ae270f98c92c80947549e1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-d, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:30:25.055 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:30:25 smithi159 podman[70525]: 2024-09-19 07:30:25.055554386 +0000 UTC m=+0.448649160 container init 637c6bb5df7e439b32a6d5072ac29026a1e734af8ae270f98c92c80947549e1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-d, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T07:30:25.387 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:30:25 smithi159 podman[70525]: 2024-09-19 07:30:25.061734439 +0000 UTC m=+0.454829189 container start 637c6bb5df7e439b32a6d5072ac29026a1e734af8ae270f98c92c80947549e1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-d, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T07:30:25.387 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:30:25 smithi159 ceph-mds[70544]: -- 172.21.15.159:0/380650174 <== mon.0 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55be8a7c7860 con 0x55be8a7a5000 2024-09-19T07:30:25.387 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:30:25 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-d[70540]: starting mds.d at 2024-09-19T07:30:25.387 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:30:25 smithi159 bash[70525]: 637c6bb5df7e439b32a6d5072ac29026a1e734af8ae270f98c92c80947549e1b 2024-09-19T07:30:25.388 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:30:25 smithi159 systemd[1]: Started Ceph mds.d for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:30:25.974 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:25 smithi189 ceph-mon[34851]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:25.974 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:25 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:25.974 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:25 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:25.974 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:25 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:25.974 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:25 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:25.974 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:25 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:25.974 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:25 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:26.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:25 smithi203 ceph-mon[35335]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:26.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:25 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:26.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:25 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:26.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:25 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:26.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:25 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:26.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:25 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:26.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:25 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:26.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:25 smithi159 ceph-mon[30472]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:26.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:25 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:26.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:25 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:26.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:25 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:26.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:25 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:26.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:25 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:26.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:25 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:26.137 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:30:26.511 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:27.034 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:26 smithi159 ceph-mon[30472]: Deploying daemon mds.e on smithi189 2024-09-19T07:30:27.034 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:26 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.159:6836/3464449771,v1:172.21.15.159:6837/3464449771] up:boot 2024-09-19T07:30:27.035 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:26 smithi159 ceph-mon[30472]: fsmap 4 up:standby 2024-09-19T07:30:27.035 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:26 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T07:30:27.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:26 smithi203 ceph-mon[35335]: Deploying daemon mds.e on smithi189 2024-09-19T07:30:27.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:26 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.159:6836/3464449771,v1:172.21.15.159:6837/3464449771] up:boot 2024-09-19T07:30:27.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:26 smithi203 ceph-mon[35335]: fsmap 4 up:standby 2024-09-19T07:30:27.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:26 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T07:30:27.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:26 smithi189 ceph-mon[34851]: Deploying daemon mds.e on smithi189 2024-09-19T07:30:27.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:26 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.159:6836/3464449771,v1:172.21.15.159:6837/3464449771] up:boot 2024-09-19T07:30:27.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:26 smithi189 ceph-mon[34851]: fsmap 4 up:standby 2024-09-19T07:30:27.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:26 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T07:30:27.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.144+0000 7f44cba00640 1 -- 172.21.15.159:0/2549196103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c410e3c0 msgr2=0x7f44c4110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:27.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.144+0000 7f44cba00640 1 --2- 172.21.15.159:0/2549196103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c410e3c0 0x7f44c4110810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009ca0 tx=0x7f44bc02a950 comp rx=0 tx=0).stop 2024-09-19T07:30:27.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.145+0000 7f44cba00640 1 -- 172.21.15.159:0/2549196103 shutdown_connections 2024-09-19T07:30:27.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.145+0000 7f44cba00640 1 --2- 172.21.15.159:0/2549196103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c410e3c0 0x7f44c4110810 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:27.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.145+0000 7f44cba00640 1 --2- 172.21.15.159:0/2549196103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44c4074f40 0x7f44c410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:27.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.145+0000 7f44cba00640 1 --2- 172.21.15.159:0/2549196103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44c4074620 0x7f44c4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:27.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.145+0000 7f44cba00640 1 -- 172.21.15.159:0/2549196103 >> 172.21.15.159:0/2549196103 conn(0x7f44c40fd8c0 msgr2=0x7f44c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:27.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.145+0000 7f44cba00640 1 -- 172.21.15.159:0/2549196103 shutdown_connections 2024-09-19T07:30:27.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.145+0000 7f44cba00640 1 -- 172.21.15.159:0/2549196103 wait complete. 2024-09-19T07:30:27.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.146+0000 7f44cba00640 1 Processor -- start 2024-09-19T07:30:27.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.146+0000 7f44cba00640 1 -- start start 2024-09-19T07:30:27.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.146+0000 7f44cba00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44c4074620 0x7f44c419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:27.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.146+0000 7f44cba00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44c4074f40 0x7f44c419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:27.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.146+0000 7f44cba00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c410e3c0 0x7f44c419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:27.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.146+0000 7f44cba00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f44c41141a0 con 0x7f44c410e3c0 2024-09-19T07:30:27.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.146+0000 7f44cba00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4114020 con 0x7f44c4074620 2024-09-19T07:30:27.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.146+0000 7f44cba00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4114320 con 0x7f44c4074f40 2024-09-19T07:30:27.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.146+0000 7f44c9000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44c4074620 0x7f44c419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:27.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.146+0000 7f44c9000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44c4074620 0x7f44c419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:51668/0 (socket says 172.21.15.159:51668) 2024-09-19T07:30:27.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.146+0000 7f44c9000640 1 -- 172.21.15.159:0/931964024 learned_addr learned my addr 172.21.15.159:0/931964024 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:30:27.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.146+0000 7f44bbe00640 1 --2- 172.21.15.159:0/931964024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44c4074f40 0x7f44c419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:27.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.146+0000 7f44c9a00640 1 --2- 172.21.15.159:0/931964024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c410e3c0 0x7f44c419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:27.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.147+0000 7f44c9000640 1 -- 172.21.15.159:0/931964024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44c4074f40 msgr2=0x7f44c419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:27.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.147+0000 7f44c9000640 1 --2- 172.21.15.159:0/931964024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44c4074f40 0x7f44c419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:27.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.147+0000 7f44c9000640 1 -- 172.21.15.159:0/931964024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c410e3c0 msgr2=0x7f44c419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:27.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.147+0000 7f44c9000640 1 --2- 172.21.15.159:0/931964024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c410e3c0 0x7f44c419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:27.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.147+0000 7f44c9000640 1 -- 172.21.15.159:0/931964024 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c419f5d0 con 0x7f44c4074620 2024-09-19T07:30:27.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.147+0000 7f44c9a00640 1 --2- 172.21.15.159:0/931964024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c410e3c0 0x7f44c419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:30:27.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.147+0000 7f44c9000640 1 --2- 172.21.15.159:0/931964024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44c4074620 0x7f44c419a620 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f44b400d950 tx=0x7f44b400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:27.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.147+0000 7f44c2000640 1 -- 172.21.15.159:0/931964024 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4017030 con 0x7f44c4074620 2024-09-19T07:30:27.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.147+0000 7f44cba00640 1 -- 172.21.15.159:0/931964024 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c419f8c0 con 0x7f44c4074620 2024-09-19T07:30:27.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.147+0000 7f44c2000640 1 -- 172.21.15.159:0/931964024 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b4004020 con 0x7f44c4074620 2024-09-19T07:30:27.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.147+0000 7f44c2000640 1 -- 172.21.15.159:0/931964024 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b40047a0 con 0x7f44c4074620 2024-09-19T07:30:27.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.147+0000 7f44cba00640 1 -- 172.21.15.159:0/931964024 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c41a71a0 con 0x7f44c4074620 2024-09-19T07:30:27.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.148+0000 7f44cba00640 1 -- 172.21.15.159:0/931964024 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c4074a00 con 0x7f44c4074620 2024-09-19T07:30:27.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.150+0000 7f44c2000640 1 -- 172.21.15.159:0/931964024 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f44b40041d0 con 0x7f44c4074620 2024-09-19T07:30:27.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.151+0000 7f44c2000640 1 --2- 172.21.15.159:0/931964024 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f44a80a32c0 0x7f44a80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:27.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.152+0000 7f44c2000640 1 -- 172.21.15.159:0/931964024 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f44b40cee80 con 0x7f44c4074620 2024-09-19T07:30:27.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.152+0000 7f44bbe00640 1 --2- 172.21.15.159:0/931964024 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f44a80a32c0 0x7f44a80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:27.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.152+0000 7f44bbe00640 1 --2- 172.21.15.159:0/931964024 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f44a80a32c0 0x7f44a80a5780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f44c410f6e0 tx=0x7f44ac005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:27.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.153+0000 7f44c2000640 1 -- 172.21.15.159:0/931964024 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f44b409c0b0 con 0x7f44c4074620 2024-09-19T07:30:27.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.362+0000 7f44cba00640 1 -- 172.21.15.159:0/931964024 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44c419b980 con 0x7f44c4074620 2024-09-19T07:30:27.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.362+0000 7f44c2000640 1 -- 172.21.15.159:0/931964024 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3919 (secure 0 0 0) 0x7f44b409b9b0 con 0x7f44c4074620 2024-09-19T07:30:27.364 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:30:27.364 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":5,"btime":"2024-09-19T07:30:25:766627+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3464449771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3464449771},{"type":"v1","addr":"172.21.15.159:6837","nonce":3464449771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-19T07:30:27.364 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 5 2024-09-19T07:30:27.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.365+0000 7f44cba00640 1 -- 172.21.15.159:0/931964024 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f44a80a32c0 msgr2=0x7f44a80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:27.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.365+0000 7f44cba00640 1 --2- 172.21.15.159:0/931964024 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f44a80a32c0 0x7f44a80a5780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f44c410f6e0 tx=0x7f44ac005f90 comp rx=0 tx=0).stop 2024-09-19T07:30:27.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.365+0000 7f44cba00640 1 -- 172.21.15.159:0/931964024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44c4074620 msgr2=0x7f44c419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:27.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.365+0000 7f44cba00640 1 --2- 172.21.15.159:0/931964024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44c4074620 0x7f44c419a620 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f44b400d950 tx=0x7f44b400de10 comp rx=0 tx=0).stop 2024-09-19T07:30:27.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.365+0000 7f44cba00640 1 -- 172.21.15.159:0/931964024 shutdown_connections 2024-09-19T07:30:27.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.365+0000 7f44cba00640 1 --2- 172.21.15.159:0/931964024 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f44a80a32c0 0x7f44a80a5780 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:27.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.365+0000 7f44cba00640 1 --2- 172.21.15.159:0/931964024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c410e3c0 0x7f44c419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:27.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.365+0000 7f44cba00640 1 --2- 172.21.15.159:0/931964024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44c4074f40 0x7f44c419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:27.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.365+0000 7f44cba00640 1 --2- 172.21.15.159:0/931964024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44c4074620 0x7f44c419a620 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:27.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.365+0000 7f44cba00640 1 -- 172.21.15.159:0/931964024 >> 172.21.15.159:0/931964024 conn(0x7f44c40fd8c0 msgr2=0x7f44c40fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:27.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.365+0000 7f44cba00640 1 -- 172.21.15.159:0/931964024 shutdown_connections 2024-09-19T07:30:27.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:27.365+0000 7f44cba00640 1 -- 172.21.15.159:0/931964024 wait complete. 2024-09-19T07:30:27.556 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:30:27 smithi189 systemd[1]: Starting Ceph mds.e for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:30:27.826 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:27 smithi189 ceph-mon[34851]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:27.826 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:27 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/931964024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:27.827 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:30:27 smithi189 podman[64074]: 2024-09-19 07:30:27.464945191 +0000 UTC m=+0.015948360 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:30:27.827 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:30:27 smithi189 podman[64074]: 2024-09-19 07:30:27.581936964 +0000 UTC m=+0.132940127 container create aa4b2d77bfa0e76f040f12c4e626b60532b4b6d8ff41d072311ff571f1896856 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-19T07:30:27.827 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:30:27 smithi189 podman[64074]: 2024-09-19 07:30:27.82697891 +0000 UTC m=+0.377982075 container init aa4b2d77bfa0e76f040f12c4e626b60532b4b6d8ff41d072311ff571f1896856 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T07:30:28.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:27 smithi159 ceph-mon[30472]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:27 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/931964024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:28.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:27 smithi203 ceph-mon[35335]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:28.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:27 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/931964024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:28.305 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:30:27 smithi189 podman[64074]: 2024-09-19 07:30:27.829943602 +0000 UTC m=+0.380946767 container start aa4b2d77bfa0e76f040f12c4e626b60532b4b6d8ff41d072311ff571f1896856 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-19T07:30:28.306 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:30:27 smithi189 ceph-mds[64094]: -- 172.21.15.189:0/2169294376 <== mon.2 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560976d29c20 con 0x560976d07000 2024-09-19T07:30:28.306 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:30:27 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-e[64090]: starting mds.e at 2024-09-19T07:30:28.306 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:30:27 smithi189 bash[64074]: aa4b2d77bfa0e76f040f12c4e626b60532b4b6d8ff41d072311ff571f1896856 2024-09-19T07:30:28.306 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:30:27 smithi189 systemd[1]: Started Ceph mds.e for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:30:29.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:28 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:29.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:28 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:29.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:28 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:29.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:28 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:29.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:28 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:29.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:28 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:29.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:28 smithi203 ceph-mon[35335]: Deploying daemon mds.f on smithi203 2024-09-19T07:30:29.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:28 smithi203 ceph-mon[35335]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:29.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:28 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.189:6834/1340279549,v1:172.21.15.189:6835/1340279549] up:boot 2024-09-19T07:30:29.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:28 smithi203 ceph-mon[35335]: fsmap 5 up:standby 2024-09-19T07:30:29.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:28 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:30:29.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:28 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:29.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:28 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:29.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:28 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:29.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:28 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:29.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:28 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:29.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:28 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:29.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:28 smithi189 ceph-mon[34851]: Deploying daemon mds.f on smithi203 2024-09-19T07:30:29.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:28 smithi189 ceph-mon[34851]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:29.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:28 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.189:6834/1340279549,v1:172.21.15.189:6835/1340279549] up:boot 2024-09-19T07:30:29.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:28 smithi189 ceph-mon[34851]: fsmap 5 up:standby 2024-09-19T07:30:29.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:28 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:30:29.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:28 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:29.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:28 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:29.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:28 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:29.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:28 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:29.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:28 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:29.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:28 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:29.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:28 smithi159 ceph-mon[30472]: Deploying daemon mds.f on smithi203 2024-09-19T07:30:29.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:28 smithi159 ceph-mon[30472]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:29.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:28 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.189:6834/1340279549,v1:172.21.15.189:6835/1340279549] up:boot 2024-09-19T07:30:29.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:28 smithi159 ceph-mon[30472]: fsmap 5 up:standby 2024-09-19T07:30:29.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:28 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:30:30.195 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:30:30.238 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:30:30 smithi203 systemd[1]: Starting Ceph mds.f for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:30:30.532 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:30:30 smithi203 podman[65162]: 2024-09-19 07:30:30.14029318 +0000 UTC m=+0.015797524 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:30:30.532 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:30:30 smithi203 podman[65162]: 2024-09-19 07:30:30.263297961 +0000 UTC m=+0.138802350 container create fc71e70c06dd2955171cb692fc7b07f8d74d422f69e071c9873f376acb49a22b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True) 2024-09-19T07:30:30.532 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:30:30 smithi203 podman[65162]: 2024-09-19 07:30:30.532575409 +0000 UTC m=+0.408079752 container init fc71e70c06dd2955171cb692fc7b07f8d74d422f69e071c9873f376acb49a22b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T07:30:30.575 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:30.786 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:30:30 smithi203 podman[65162]: 2024-09-19 07:30:30.535388252 +0000 UTC m=+0.410892596 container start fc71e70c06dd2955171cb692fc7b07f8d74d422f69e071c9873f376acb49a22b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T07:30:30.786 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:30:30 smithi203 ceph-mds[65181]: -- 172.21.15.203:0/730701755 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555ce31c7860 con 0x555ce31a5000 2024-09-19T07:30:30.786 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:30:30 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-f[65177]: starting mds.f at 2024-09-19T07:30:30.786 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:30:30 smithi203 bash[65162]: fc71e70c06dd2955171cb692fc7b07f8d74d422f69e071c9873f376acb49a22b 2024-09-19T07:30:30.786 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:30:30 smithi203 systemd[1]: Started Ceph mds.f for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:30:31.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.127+0000 7fa7bc200640 1 -- 172.21.15.159:0/1891390089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7b4074f40 msgr2=0x7fa7b410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:31.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.127+0000 7fa7bc200640 1 --2- 172.21.15.159:0/1891390089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7b4074f40 0x7fa7b410de80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009900 tx=0x7fa7a002a480 comp rx=0 tx=0).stop 2024-09-19T07:30:31.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.127+0000 7fa7bc200640 1 -- 172.21.15.159:0/1891390089 shutdown_connections 2024-09-19T07:30:31.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.127+0000 7fa7bc200640 1 --2- 172.21.15.159:0/1891390089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7b410e3c0 0x7fa7b4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:31.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.127+0000 7fa7bc200640 1 --2- 172.21.15.159:0/1891390089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7b4074f40 0x7fa7b410de80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:31.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.127+0000 7fa7bc200640 1 --2- 172.21.15.159:0/1891390089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7b4074620 0x7fa7b4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:31.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.127+0000 7fa7bc200640 1 -- 172.21.15.159:0/1891390089 >> 172.21.15.159:0/1891390089 conn(0x7fa7b40fd8c0 msgr2=0x7fa7b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:31.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.127+0000 7fa7bc200640 1 -- 172.21.15.159:0/1891390089 shutdown_connections 2024-09-19T07:30:31.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.127+0000 7fa7bc200640 1 -- 172.21.15.159:0/1891390089 wait complete. 2024-09-19T07:30:31.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.128+0000 7fa7bc200640 1 Processor -- start 2024-09-19T07:30:31.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.128+0000 7fa7bc200640 1 -- start start 2024-09-19T07:30:31.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.128+0000 7fa7bc200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7b4074620 0x7fa7b419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:31.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.128+0000 7fa7bc200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7b4074f40 0x7fa7b419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:31.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.128+0000 7fa7bc200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7b410e3c0 0x7fa7b419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:31.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.128+0000 7fa7bc200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4114160 con 0x7fa7b4074f40 2024-09-19T07:30:31.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.128+0000 7fa7bc200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4113fe0 con 0x7fa7b4074620 2024-09-19T07:30:31.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.128+0000 7fa7bc200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41142e0 con 0x7fa7b410e3c0 2024-09-19T07:30:31.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.129+0000 7fa7b8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7b4074f40 0x7fa7b419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.129+0000 7fa7b8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7b4074f40 0x7fa7b419abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47160/0 (socket says 172.21.15.159:47160) 2024-09-19T07:30:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.129+0000 7fa7b9800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7b4074620 0x7fa7b419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.129+0000 7fa7ba200640 1 --2- 172.21.15.159:0/2279995505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7b410e3c0 0x7fa7b419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.129+0000 7fa7b8e00640 1 -- 172.21.15.159:0/2279995505 learned_addr learned my addr 172.21.15.159:0/2279995505 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:30:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.129+0000 7fa7ba200640 1 -- 172.21.15.159:0/2279995505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7b4074620 msgr2=0x7fa7b419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.129+0000 7fa7ba200640 1 --2- 172.21.15.159:0/2279995505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7b4074620 0x7fa7b419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.129+0000 7fa7ba200640 1 -- 172.21.15.159:0/2279995505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7b4074f40 msgr2=0x7fa7b419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.129+0000 7fa7ba200640 1 --2- 172.21.15.159:0/2279995505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7b4074f40 0x7fa7b419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.129+0000 7fa7ba200640 1 -- 172.21.15.159:0/2279995505 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b419f630 con 0x7fa7b410e3c0 2024-09-19T07:30:31.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.129+0000 7fa7ba200640 1 --2- 172.21.15.159:0/2279995505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7b410e3c0 0x7fa7b419ef50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00e970 tx=0x7fa7ac00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:31.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.129+0000 7fa7b2000640 1 -- 172.21.15.159:0/2279995505 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac00ccc0 con 0x7fa7b410e3c0 2024-09-19T07:30:31.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.130+0000 7fa7bc200640 1 -- 172.21.15.159:0/2279995505 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b419f920 con 0x7fa7b410e3c0 2024-09-19T07:30:31.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.130+0000 7fa7b2000640 1 -- 172.21.15.159:0/2279995505 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7ac005020 con 0x7fa7b410e3c0 2024-09-19T07:30:31.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.130+0000 7fa7b2000640 1 -- 172.21.15.159:0/2279995505 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac004a00 con 0x7fa7b410e3c0 2024-09-19T07:30:31.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.130+0000 7fa7bc200640 1 -- 172.21.15.159:0/2279995505 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b41a7200 con 0x7fa7b410e3c0 2024-09-19T07:30:31.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.131+0000 7fa7bc200640 1 -- 172.21.15.159:0/2279995505 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b4074a00 con 0x7fa7b410e3c0 2024-09-19T07:30:31.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.132+0000 7fa7b2000640 1 -- 172.21.15.159:0/2279995505 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa7ac0051d0 con 0x7fa7b410e3c0 2024-09-19T07:30:31.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.134+0000 7fa7b2000640 1 --2- 172.21.15.159:0/2279995505 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa79c0a3240 0x7fa79c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:31.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.134+0000 7fa7b2000640 1 -- 172.21.15.159:0/2279995505 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa7ac0cf400 con 0x7fa7b410e3c0 2024-09-19T07:30:31.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.134+0000 7fa7b9800640 1 --2- 172.21.15.159:0/2279995505 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa79c0a3240 0x7fa79c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:31.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.134+0000 7fa7b9800640 1 --2- 172.21.15.159:0/2279995505 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa79c0a3240 0x7fa79c0a5700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa7a80099a0 tx=0x7fa7a8009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:31.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.139+0000 7fa7b2000640 1 -- 172.21.15.159:0/2279995505 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa7ac09c520 con 0x7fa7b410e3c0 2024-09-19T07:30:31.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.351+0000 7fa7bc200640 1 -- 172.21.15.159:0/2279995505 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7b402d7c0 con 0x7fa7b410e3c0 2024-09-19T07:30:31.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.352+0000 7fa7b2000640 1 -- 172.21.15.159:0/2279995505 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5572 (secure 0 0 0) 0x7fa7ac09be20 con 0x7fa7b410e3c0 2024-09-19T07:30:31.353 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:30:31.353 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":7,"btime":"2024-09-19T07:30:30:613974+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3464449771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3464449771},{"type":"v1","addr":"172.21.15.159:6837","nonce":3464449771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1340279549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1340279549},{"type":"v1","addr":"172.21.15.189:6835","nonce":1340279549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/937970878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":937970878},{"type":"v1","addr":"172.21.15.203:6835","nonce":937970878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-19T07:30:31.353 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 7 2024-09-19T07:30:31.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.354+0000 7fa7bc200640 1 -- 172.21.15.159:0/2279995505 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa79c0a3240 msgr2=0x7fa79c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:31.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.354+0000 7fa7bc200640 1 --2- 172.21.15.159:0/2279995505 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa79c0a3240 0x7fa79c0a5700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa7a80099a0 tx=0x7fa7a8009500 comp rx=0 tx=0).stop 2024-09-19T07:30:31.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.354+0000 7fa7bc200640 1 -- 172.21.15.159:0/2279995505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7b410e3c0 msgr2=0x7fa7b419ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:31.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.354+0000 7fa7bc200640 1 --2- 172.21.15.159:0/2279995505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7b410e3c0 0x7fa7b419ef50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00e970 tx=0x7fa7ac00ee30 comp rx=0 tx=0).stop 2024-09-19T07:30:31.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.355+0000 7fa7bc200640 1 -- 172.21.15.159:0/2279995505 shutdown_connections 2024-09-19T07:30:31.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.355+0000 7fa7bc200640 1 --2- 172.21.15.159:0/2279995505 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa79c0a3240 0x7fa79c0a5700 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:31.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.355+0000 7fa7bc200640 1 --2- 172.21.15.159:0/2279995505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7b410e3c0 0x7fa7b419ef50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:31.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.355+0000 7fa7bc200640 1 --2- 172.21.15.159:0/2279995505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7b4074f40 0x7fa7b419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:31.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.355+0000 7fa7bc200640 1 --2- 172.21.15.159:0/2279995505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7b4074620 0x7fa7b419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:31.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.355+0000 7fa7bc200640 1 -- 172.21.15.159:0/2279995505 >> 172.21.15.159:0/2279995505 conn(0x7fa7b40fd8c0 msgr2=0x7fa7b40fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:31.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.355+0000 7fa7bc200640 1 -- 172.21.15.159:0/2279995505 shutdown_connections 2024-09-19T07:30:31.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:31.355+0000 7fa7bc200640 1 -- 172.21.15.159:0/2279995505 wait complete. 2024-09-19T07:30:31.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:31 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.203:6834/937970878,v1:172.21.15.203:6835/937970878] up:boot 2024-09-19T07:30:31.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:31 smithi203 ceph-mon[35335]: fsmap 6 up:standby 2024-09-19T07:30:31.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:30:31.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:31 smithi203 ceph-mon[35335]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:31.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:31.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:31.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:31.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:31.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:31.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:31.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:31 smithi203 ceph-mon[35335]: Deploying daemon mds.c on smithi203 2024-09-19T07:30:31.787 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:31 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/2279995505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:31.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:31 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.203:6834/937970878,v1:172.21.15.203:6835/937970878] up:boot 2024-09-19T07:30:31.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:31 smithi159 ceph-mon[30472]: fsmap 6 up:standby 2024-09-19T07:30:31.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:30:31.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:31 smithi159 ceph-mon[30472]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:31.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:31.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:31.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:31.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:31.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:31.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:31.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:31 smithi159 ceph-mon[30472]: Deploying daemon mds.c on smithi203 2024-09-19T07:30:31.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:31 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2279995505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:32.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:31 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.203:6834/937970878,v1:172.21.15.203:6835/937970878] up:boot 2024-09-19T07:30:32.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:31 smithi189 ceph-mon[34851]: fsmap 6 up:standby 2024-09-19T07:30:32.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:30:32.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:31 smithi189 ceph-mon[34851]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:32.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:32.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:32.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:32.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:32.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:32.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:32.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:31 smithi189 ceph-mon[34851]: Deploying daemon mds.c on smithi203 2024-09-19T07:30:32.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:31 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/2279995505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:32.786 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:30:32 smithi203 systemd[1]: Starting Ceph mds.c for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:30:33.113 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:30:32 smithi203 podman[65464]: 2024-09-19 07:30:32.760246471 +0000 UTC m=+0.015717558 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:30:33.113 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:30:32 smithi203 podman[65464]: 2024-09-19 07:30:32.868973257 +0000 UTC m=+0.124444372 container create 1e80ff8e50531c7aa0101a6a4f85cdd749497210d6c14d93b3ab1483736aa293 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-19T07:30:33.113 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:30:33 smithi203 podman[65464]: 2024-09-19 07:30:33.11353522 +0000 UTC m=+0.369006293 container init 1e80ff8e50531c7aa0101a6a4f85cdd749497210d6c14d93b3ab1483736aa293 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T07:30:33.536 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:30:33 smithi203 podman[65464]: 2024-09-19 07:30:33.116523015 +0000 UTC m=+0.371994097 container start 1e80ff8e50531c7aa0101a6a4f85cdd749497210d6c14d93b3ab1483736aa293 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-c, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-19T07:30:33.536 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:30:33 smithi203 ceph-mds[65483]: -- 172.21.15.203:0/2728103348 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56418f6e1860 con 0x56418f6bf400 2024-09-19T07:30:33.536 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:30:33 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-c[65479]: starting mds.c at 2024-09-19T07:30:33.536 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:30:33 smithi203 bash[65464]: 1e80ff8e50531c7aa0101a6a4f85cdd749497210d6c14d93b3ab1483736aa293 2024-09-19T07:30:33.536 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:30:33 smithi203 systemd[1]: Started Ceph mds.c for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:30:34.005 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:33 smithi189 ceph-mon[34851]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:34.005 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:34.005 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:34.005 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:34.005 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:34.005 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:34.005 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:33 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:33 smithi203 ceph-mon[35335]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:34.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:34.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:33 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:34.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:33 smithi159 ceph-mon[30472]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:34.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:34.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:34.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:34.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:34.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:34.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:33 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:34.138 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:30:34.519 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:34.958 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:34 smithi159 ceph-mon[30472]: Deploying daemon mds.h on smithi189 2024-09-19T07:30:34.958 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:34 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.203:6836/1053443810,v1:172.21.15.203:6837/1053443810] up:boot 2024-09-19T07:30:34.958 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:34 smithi159 ceph-mon[30472]: fsmap 7 up:standby 2024-09-19T07:30:34.959 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:34 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:30:35.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:34 smithi203 ceph-mon[35335]: Deploying daemon mds.h on smithi189 2024-09-19T07:30:35.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:34 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.203:6836/1053443810,v1:172.21.15.203:6837/1053443810] up:boot 2024-09-19T07:30:35.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:34 smithi203 ceph-mon[35335]: fsmap 7 up:standby 2024-09-19T07:30:35.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:34 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:30:35.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:34 smithi189 ceph-mon[34851]: Deploying daemon mds.h on smithi189 2024-09-19T07:30:35.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:34 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.203:6836/1053443810,v1:172.21.15.203:6837/1053443810] up:boot 2024-09-19T07:30:35.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:34 smithi189 ceph-mon[34851]: fsmap 7 up:standby 2024-09-19T07:30:35.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:34 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:30:35.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.068+0000 7f9f6ac00640 1 -- 172.21.15.159:0/3028364203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f64076720 msgr2=0x7f9f64076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:35.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.068+0000 7f9f6ac00640 1 --2- 172.21.15.159:0/3028364203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f64076720 0x7f9f64076b00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009860 tx=0x7f9f5802a340 comp rx=0 tx=0).stop 2024-09-19T07:30:35.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.068+0000 7f9f6ac00640 1 -- 172.21.15.159:0/3028364203 shutdown_connections 2024-09-19T07:30:35.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.068+0000 7f9f6ac00640 1 --2- 172.21.15.159:0/3028364203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f64112870 0x7f9f64114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:35.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.068+0000 7f9f6ac00640 1 --2- 172.21.15.159:0/3028364203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f64077040 0x7f9f64112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:35.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.068+0000 7f9f6ac00640 1 --2- 172.21.15.159:0/3028364203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f64076720 0x7f9f64076b00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:35.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.068+0000 7f9f6ac00640 1 -- 172.21.15.159:0/3028364203 >> 172.21.15.159:0/3028364203 conn(0x7f9f640ff8d0 msgr2=0x7f9f64101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:35.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.068+0000 7f9f6ac00640 1 -- 172.21.15.159:0/3028364203 shutdown_connections 2024-09-19T07:30:35.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.068+0000 7f9f6ac00640 1 -- 172.21.15.159:0/3028364203 wait complete. 2024-09-19T07:30:35.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.070+0000 7f9f6ac00640 1 Processor -- start 2024-09-19T07:30:35.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.070+0000 7f9f6ac00640 1 -- start start 2024-09-19T07:30:35.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.070+0000 7f9f6ac00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f64076720 0x7f9f641a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:35.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.070+0000 7f9f6ac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f64077040 0x7f9f641a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:35.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.070+0000 7f9f6ac00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f64112870 0x7f9f6419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:35.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.070+0000 7f9f6ac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9f64118400 con 0x7f9f64077040 2024-09-19T07:30:35.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.070+0000 7f9f6ac00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9f64118280 con 0x7f9f64112870 2024-09-19T07:30:35.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.070+0000 7f9f6ac00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f64118580 con 0x7f9f64076720 2024-09-19T07:30:35.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.070+0000 7f9f68e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f64077040 0x7f9f641a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:35.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.070+0000 7f9f69800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f64076720 0x7f9f641a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:35.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.070+0000 7f9f6a200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f64112870 0x7f9f6419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:35.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.070+0000 7f9f68e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f64077040 0x7f9f641a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35322/0 (socket says 172.21.15.159:35322) 2024-09-19T07:30:35.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.070+0000 7f9f68e00640 1 -- 172.21.15.159:0/1528823497 learned_addr learned my addr 172.21.15.159:0/1528823497 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:30:35.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.070+0000 7f9f69800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f64076720 0x7f9f641a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:36818/0 (socket says 172.21.15.159:36818) 2024-09-19T07:30:35.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.071+0000 7f9f69800640 1 -- 172.21.15.159:0/1528823497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f64112870 msgr2=0x7f9f6419e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:35.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.071+0000 7f9f69800640 1 --2- 172.21.15.159:0/1528823497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f64112870 0x7f9f6419e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:35.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.071+0000 7f9f69800640 1 -- 172.21.15.159:0/1528823497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f64077040 msgr2=0x7f9f641a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:35.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.071+0000 7f9f69800640 1 --2- 172.21.15.159:0/1528823497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f64077040 0x7f9f641a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.071+0000 7f9f69800640 1 -- 172.21.15.159:0/1528823497 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6419efe0 con 0x7f9f64076720 2024-09-19T07:30:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.071+0000 7f9f6a200640 1 --2- 172.21.15.159:0/1528823497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f64112870 0x7f9f6419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:30:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.071+0000 7f9f69800640 1 --2- 172.21.15.159:0/1528823497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f64076720 0x7f9f641a46a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9f58035040 tx=0x7f9f58033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.071+0000 7f9f62000640 1 -- 172.21.15.159:0/1528823497 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5803f020 con 0x7f9f64076720 2024-09-19T07:30:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.071+0000 7f9f6ac00640 1 -- 172.21.15.159:0/1528823497 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f6419f270 con 0x7f9f64076720 2024-09-19T07:30:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.072+0000 7f9f62000640 1 -- 172.21.15.159:0/1528823497 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f58030550 con 0x7f9f64076720 2024-09-19T07:30:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.072+0000 7f9f62000640 1 -- 172.21.15.159:0/1528823497 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f58037040 con 0x7f9f64076720 2024-09-19T07:30:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.072+0000 7f9f6ac00640 1 -- 172.21.15.159:0/1528823497 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f641ab450 con 0x7f9f64076720 2024-09-19T07:30:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.073+0000 7f9f6ac00640 1 -- 172.21.15.159:0/1528823497 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f64077e60 con 0x7f9f64076720 2024-09-19T07:30:35.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.075+0000 7f9f62000640 1 -- 172.21.15.159:0/1528823497 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9f58042020 con 0x7f9f64076720 2024-09-19T07:30:35.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.076+0000 7f9f62000640 1 --2- 172.21.15.159:0/1528823497 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9f4c0a3240 0x7f9f4c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:35.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.076+0000 7f9f62000640 1 -- 172.21.15.159:0/1528823497 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9f580ef5f0 con 0x7f9f64076720 2024-09-19T07:30:35.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.076+0000 7f9f68e00640 1 --2- 172.21.15.159:0/1528823497 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9f4c0a3240 0x7f9f4c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:35.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.077+0000 7f9f68e00640 1 --2- 172.21.15.159:0/1528823497 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9f4c0a3240 0x7f9f4c0a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9f6419ffc0 tx=0x7f9f50009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:35.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.078+0000 7f9f62000640 1 -- 172.21.15.159:0/1528823497 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9f58049020 con 0x7f9f64076720 2024-09-19T07:30:35.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.290+0000 7f9f6ac00640 1 -- 172.21.15.159:0/1528823497 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f6410da00 con 0x7f9f64076720 2024-09-19T07:30:35.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.291+0000 7f9f62000640 1 -- 172.21.15.159:0/1528823497 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6400 (secure 0 0 0) 0x7f9f580bc710 con 0x7f9f64076720 2024-09-19T07:30:35.292 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:30:35.292 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":8,"btime":"2024-09-19T07:30:33:692307+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3464449771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3464449771},{"type":"v1","addr":"172.21.15.159:6837","nonce":3464449771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1340279549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1340279549},{"type":"v1","addr":"172.21.15.189:6835","nonce":1340279549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/937970878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":937970878},{"type":"v1","addr":"172.21.15.203:6835","nonce":937970878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1053443810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1053443810},{"type":"v1","addr":"172.21.15.203:6837","nonce":1053443810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T07:30:35.292 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 8 2024-09-19T07:30:35.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.293+0000 7f9f6ac00640 1 -- 172.21.15.159:0/1528823497 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9f4c0a3240 msgr2=0x7f9f4c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:35.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.293+0000 7f9f6ac00640 1 --2- 172.21.15.159:0/1528823497 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9f4c0a3240 0x7f9f4c0a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9f6419ffc0 tx=0x7f9f50009260 comp rx=0 tx=0).stop 2024-09-19T07:30:35.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.293+0000 7f9f6ac00640 1 -- 172.21.15.159:0/1528823497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f64076720 msgr2=0x7f9f641a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:35.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.293+0000 7f9f6ac00640 1 --2- 172.21.15.159:0/1528823497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f64076720 0x7f9f641a46a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9f58035040 tx=0x7f9f58033b60 comp rx=0 tx=0).stop 2024-09-19T07:30:35.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.293+0000 7f9f6ac00640 1 -- 172.21.15.159:0/1528823497 shutdown_connections 2024-09-19T07:30:35.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.293+0000 7f9f6ac00640 1 --2- 172.21.15.159:0/1528823497 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f9f4c0a3240 0x7f9f4c0a5700 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:35.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.293+0000 7f9f6ac00640 1 --2- 172.21.15.159:0/1528823497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f64112870 0x7f9f6419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:35.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.293+0000 7f9f6ac00640 1 --2- 172.21.15.159:0/1528823497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f64077040 0x7f9f641a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:35.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.293+0000 7f9f6ac00640 1 --2- 172.21.15.159:0/1528823497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f64076720 0x7f9f641a46a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:35.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.293+0000 7f9f6ac00640 1 -- 172.21.15.159:0/1528823497 >> 172.21.15.159:0/1528823497 conn(0x7f9f640ff8d0 msgr2=0x7f9f64101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:35.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.294+0000 7f9f6ac00640 1 -- 172.21.15.159:0/1528823497 shutdown_connections 2024-09-19T07:30:35.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:35.294+0000 7f9f6ac00640 1 -- 172.21.15.159:0/1528823497 wait complete. 2024-09-19T07:30:35.541 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:30:35 smithi189 systemd[1]: Starting Ceph mds.h for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:30:35.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:35 smithi189 ceph-mon[34851]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:35.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:35 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1528823497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:35.806 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:30:35 smithi189 podman[64376]: 2024-09-19 07:30:35.444298854 +0000 UTC m=+0.015861420 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:30:35.806 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:30:35 smithi189 podman[64376]: 2024-09-19 07:30:35.579305251 +0000 UTC m=+0.150867816 container create 806c9ecaa4a36bceb933560556c8690a77e4fd014a7e4f50fd4e68f282937022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-h, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T07:30:36.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:35 smithi203 ceph-mon[35335]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:36.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:35 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1528823497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:36.080 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:35 smithi159 ceph-mon[30472]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:36.080 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:35 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1528823497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:36.306 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:30:35 smithi189 podman[64376]: 2024-09-19 07:30:35.840710575 +0000 UTC m=+0.412273137 container init 806c9ecaa4a36bceb933560556c8690a77e4fd014a7e4f50fd4e68f282937022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T07:30:36.306 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:30:35 smithi189 podman[64376]: 2024-09-19 07:30:35.843806711 +0000 UTC m=+0.415369282 container start 806c9ecaa4a36bceb933560556c8690a77e4fd014a7e4f50fd4e68f282937022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-h, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T07:30:36.306 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:30:35 smithi189 ceph-mds[64395]: -- 172.21.15.189:0/1679612623 <== mon.2 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e448de7860 con 0x55e448dc4800 2024-09-19T07:30:36.306 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:30:35 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-h[64391]: starting mds.h at 2024-09-19T07:30:36.306 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:30:35 smithi189 bash[64376]: 806c9ecaa4a36bceb933560556c8690a77e4fd014a7e4f50fd4e68f282937022 2024-09-19T07:30:36.306 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:30:35 smithi189 systemd[1]: Started Ceph mds.h for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:30:37.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:37.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:37.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:37.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:37.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:37.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:37.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:37 smithi203 ceph-mon[35335]: Deploying daemon mds.i on smithi203 2024-09-19T07:30:37.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:37 smithi203 ceph-mon[35335]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:37.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:37 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] up:boot 2024-09-19T07:30:37.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:37 smithi203 ceph-mon[35335]: fsmap 8 up:standby 2024-09-19T07:30:37.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:37 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:30:37.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:37.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:37.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:37.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:37.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:37.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:37.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:37 smithi189 ceph-mon[34851]: Deploying daemon mds.i on smithi203 2024-09-19T07:30:37.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:37 smithi189 ceph-mon[34851]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:37.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:37 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] up:boot 2024-09-19T07:30:37.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:37 smithi189 ceph-mon[34851]: fsmap 8 up:standby 2024-09-19T07:30:37.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:37 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:30:37.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:36 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:37.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:36 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:37.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:36 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:37.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:36 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:37.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:36 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:37.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:36 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:37.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:36 smithi159 ceph-mon[30472]: Deploying daemon mds.i on smithi203 2024-09-19T07:30:37.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:36 smithi159 ceph-mon[30472]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:37.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:36 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] up:boot 2024-09-19T07:30:37.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:36 smithi159 ceph-mon[30472]: fsmap 8 up:standby 2024-09-19T07:30:37.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:36 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:30:38.117 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:30:38.251 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:30:38 smithi203 systemd[1]: Starting Ceph mds.i for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:30:38.494 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:38.520 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:30:38 smithi203 podman[65766]: 2024-09-19 07:30:38.153568896 +0000 UTC m=+0.016536181 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:30:38.520 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:30:38 smithi203 podman[65766]: 2024-09-19 07:30:38.258351141 +0000 UTC m=+0.121318419 container create 826b5b583f2d7345b1b6779772f18f337edf23344c7056e71d0d679143d2cd36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T07:30:38.521 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:30:38 smithi203 podman[65766]: 2024-09-19 07:30:38.494201866 +0000 UTC m=+0.357169145 container init 826b5b583f2d7345b1b6779772f18f337edf23344c7056e71d0d679143d2cd36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T07:30:38.521 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:30:38 smithi203 podman[65766]: 2024-09-19 07:30:38.497019812 +0000 UTC m=+0.359987089 container start 826b5b583f2d7345b1b6779772f18f337edf23344c7056e71d0d679143d2cd36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-09-19T07:30:38.786 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:30:38 smithi203 ceph-mds[65785]: -- 172.21.15.203:0/2352333480 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5607c92ab860 con 0x5607c9289400 2024-09-19T07:30:38.786 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:30:38 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-i[65781]: starting mds.i at 2024-09-19T07:30:38.786 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:30:38 smithi203 bash[65766]: 826b5b583f2d7345b1b6779772f18f337edf23344c7056e71d0d679143d2cd36 2024-09-19T07:30:38.786 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:30:38 smithi203 systemd[1]: Started Ceph mds.i for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:30:39.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.083+0000 7fa65c000640 1 -- 172.21.15.159:0/2741350880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa654103ff0 msgr2=0x7fa6541043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:39.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.083+0000 7fa65c000640 1 --2- 172.21.15.159:0/2741350880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa654103ff0 0x7fa6541043d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa648009860 tx=0x7fa64802a340 comp rx=0 tx=0).stop 2024-09-19T07:30:39.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.084+0000 7fa65c000640 1 -- 172.21.15.159:0/2741350880 shutdown_connections 2024-09-19T07:30:39.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.084+0000 7fa65c000640 1 --2- 172.21.15.159:0/2741350880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa654109120 0x7fa654110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:39.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.084+0000 7fa65c000640 1 --2- 172.21.15.159:0/2741350880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6541049a0 0x7fa6541089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:39.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.084+0000 7fa65c000640 1 --2- 172.21.15.159:0/2741350880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa654103ff0 0x7fa6541043d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:39.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.084+0000 7fa65c000640 1 -- 172.21.15.159:0/2741350880 >> 172.21.15.159:0/2741350880 conn(0x7fa6540ffa90 msgr2=0x7fa654101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:39.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.084+0000 7fa65c000640 1 -- 172.21.15.159:0/2741350880 shutdown_connections 2024-09-19T07:30:39.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.084+0000 7fa65c000640 1 -- 172.21.15.159:0/2741350880 wait complete. 2024-09-19T07:30:39.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.085+0000 7fa65c000640 1 Processor -- start 2024-09-19T07:30:39.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.085+0000 7fa65c000640 1 -- start start 2024-09-19T07:30:39.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.086+0000 7fa65c000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa654103ff0 0x7fa65419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:39.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.086+0000 7fa65c000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6541049a0 0x7fa65419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:39.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.086+0000 7fa65c000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa654109120 0x7fa65419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:39.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.086+0000 7fa65c000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa654114120 con 0x7fa6541049a0 2024-09-19T07:30:39.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.086+0000 7fa65c000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa654113fa0 con 0x7fa654103ff0 2024-09-19T07:30:39.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.086+0000 7fa65c000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa6541142a0 con 0x7fa654109120 2024-09-19T07:30:39.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.086+0000 7fa658c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6541049a0 0x7fa65419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:39.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.086+0000 7fa658c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6541049a0 0x7fa65419ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35346/0 (socket says 172.21.15.159:35346) 2024-09-19T07:30:39.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.086+0000 7fa658c00640 1 -- 172.21.15.159:0/2954904530 learned_addr learned my addr 172.21.15.159:0/2954904530 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:30:39.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.086+0000 7fa659600640 1 --2- 172.21.15.159:0/2954904530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa654103ff0 0x7fa65419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:39.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.086+0000 7fa65a000640 1 --2- 172.21.15.159:0/2954904530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa654109120 0x7fa65419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:39.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.086+0000 7fa658c00640 1 -- 172.21.15.159:0/2954904530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa654109120 msgr2=0x7fa65419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:39.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.086+0000 7fa658c00640 1 --2- 172.21.15.159:0/2954904530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa654109120 0x7fa65419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:39.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.086+0000 7fa658c00640 1 -- 172.21.15.159:0/2954904530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa654103ff0 msgr2=0x7fa65419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:39.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.086+0000 7fa658c00640 1 --2- 172.21.15.159:0/2954904530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa654103ff0 0x7fa65419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:39.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.086+0000 7fa658c00640 1 -- 172.21.15.159:0/2954904530 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65419f5d0 con 0x7fa6541049a0 2024-09-19T07:30:39.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.086+0000 7fa659600640 1 --2- 172.21.15.159:0/2954904530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa654103ff0 0x7fa65419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:30:39.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.087+0000 7fa658c00640 1 --2- 172.21.15.159:0/2954904530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6541049a0 0x7fa65419ab60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa64000e9c0 tx=0x7fa64000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:39.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.087+0000 7fa652000640 1 -- 172.21.15.159:0/2954904530 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64000cde0 con 0x7fa6541049a0 2024-09-19T07:30:39.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.087+0000 7fa652000640 1 -- 172.21.15.159:0/2954904530 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6400035f0 con 0x7fa6541049a0 2024-09-19T07:30:39.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.087+0000 7fa65c000640 1 -- 172.21.15.159:0/2954904530 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65419f8c0 con 0x7fa6541049a0 2024-09-19T07:30:39.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.087+0000 7fa652000640 1 -- 172.21.15.159:0/2954904530 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640010690 con 0x7fa6541049a0 2024-09-19T07:30:39.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.087+0000 7fa65c000640 1 -- 172.21.15.159:0/2954904530 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6541a71a0 con 0x7fa6541049a0 2024-09-19T07:30:39.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.088+0000 7fa65c000640 1 -- 172.21.15.159:0/2954904530 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6541043d0 con 0x7fa6541049a0 2024-09-19T07:30:39.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.091+0000 7fa652000640 1 -- 172.21.15.159:0/2954904530 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa6400037a0 con 0x7fa6541049a0 2024-09-19T07:30:39.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.092+0000 7fa652000640 1 --2- 172.21.15.159:0/2954904530 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa63c0a3240 0x7fa63c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:39.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.092+0000 7fa652000640 1 -- 172.21.15.159:0/2954904530 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa6400cf840 con 0x7fa6541049a0 2024-09-19T07:30:39.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.092+0000 7fa659600640 1 --2- 172.21.15.159:0/2954904530 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa63c0a3240 0x7fa63c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:39.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.093+0000 7fa659600640 1 --2- 172.21.15.159:0/2954904530 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa63c0a3240 0x7fa63c0a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa64802a930 tx=0x7fa6480056d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:39.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.097+0000 7fa652000640 1 -- 172.21.15.159:0/2954904530 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa64009c960 con 0x7fa6541049a0 2024-09-19T07:30:39.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.356+0000 7fa65c000640 1 -- 172.21.15.159:0/2954904530 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa65402d7c0 con 0x7fa6541049a0 2024-09-19T07:30:39.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.356+0000 7fa652000640 1 -- 172.21.15.159:0/2954904530 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8058 (secure 0 0 0) 0x7fa64009c260 con 0x7fa6541049a0 2024-09-19T07:30:39.358 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:30:39.358 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":10,"btime":"2024-09-19T07:30:38:576468+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3464449771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3464449771},{"type":"v1","addr":"172.21.15.159:6837","nonce":3464449771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1340279549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1340279549},{"type":"v1","addr":"172.21.15.189:6835","nonce":1340279549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/937970878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":937970878},{"type":"v1","addr":"172.21.15.203:6835","nonce":937970878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1053443810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1053443810},{"type":"v1","addr":"172.21.15.203:6837","nonce":1053443810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1030825585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1030825585},{"type":"v1","addr":"172.21.15.189:6837","nonce":1030825585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/3862781462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":3862781462},{"type":"v1","addr":"172.21.15.203:6839","nonce":3862781462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T07:30:39.358 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 10 2024-09-19T07:30:39.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.359+0000 7fa65c000640 1 -- 172.21.15.159:0/2954904530 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa63c0a3240 msgr2=0x7fa63c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:39.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.359+0000 7fa65c000640 1 --2- 172.21.15.159:0/2954904530 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa63c0a3240 0x7fa63c0a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa64802a930 tx=0x7fa6480056d0 comp rx=0 tx=0).stop 2024-09-19T07:30:39.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.359+0000 7fa65c000640 1 -- 172.21.15.159:0/2954904530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6541049a0 msgr2=0x7fa65419ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.359+0000 7fa65c000640 1 --2- 172.21.15.159:0/2954904530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6541049a0 0x7fa65419ab60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa64000e9c0 tx=0x7fa64000ee80 comp rx=0 tx=0).stop 2024-09-19T07:30:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.359+0000 7fa65c000640 1 -- 172.21.15.159:0/2954904530 shutdown_connections 2024-09-19T07:30:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.359+0000 7fa65c000640 1 --2- 172.21.15.159:0/2954904530 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa63c0a3240 0x7fa63c0a5700 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.359+0000 7fa65c000640 1 --2- 172.21.15.159:0/2954904530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa654109120 0x7fa65419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.359+0000 7fa65c000640 1 --2- 172.21.15.159:0/2954904530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6541049a0 0x7fa65419ab60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.359+0000 7fa65c000640 1 --2- 172.21.15.159:0/2954904530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa654103ff0 0x7fa65419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.359+0000 7fa65c000640 1 -- 172.21.15.159:0/2954904530 >> 172.21.15.159:0/2954904530 conn(0x7fa6540ffa90 msgr2=0x7fa654100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.360+0000 7fa65c000640 1 -- 172.21.15.159:0/2954904530 shutdown_connections 2024-09-19T07:30:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:39.360+0000 7fa65c000640 1 -- 172.21.15.159:0/2954904530 wait complete. 2024-09-19T07:30:39.613 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:39 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] up:boot 2024-09-19T07:30:39.613 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:39 smithi159 ceph-mon[30472]: fsmap 9 up:standby 2024-09-19T07:30:39.613 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:39 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:30:39.613 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:39 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:39.613 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:39 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:39.613 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:39 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:39.613 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:39 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:39.614 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:39 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:39.614 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:39 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:39.614 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:39 smithi159 ceph-mon[30472]: Deploying daemon mds.g on smithi159 2024-09-19T07:30:39.614 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:39 smithi159 ceph-mon[30472]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:39.614 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:39 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2954904530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:39 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] up:boot 2024-09-19T07:30:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:39 smithi203 ceph-mon[35335]: fsmap 9 up:standby 2024-09-19T07:30:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:39 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:30:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:39 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:39 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:39 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:39 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:40.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:39 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:40.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:39 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:40.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:39 smithi203 ceph-mon[35335]: Deploying daemon mds.g on smithi159 2024-09-19T07:30:40.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:39 smithi203 ceph-mon[35335]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:40.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:39 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/2954904530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:39 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] up:boot 2024-09-19T07:30:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:39 smithi189 ceph-mon[34851]: fsmap 9 up:standby 2024-09-19T07:30:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:39 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:30:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:39 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:39 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:39 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:39 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:39 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:39 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:39 smithi189 ceph-mon[34851]: Deploying daemon mds.g on smithi159 2024-09-19T07:30:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:39 smithi189 ceph-mon[34851]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:40.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:39 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/2954904530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:41.928 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:41 smithi159 ceph-mon[30472]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:42.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:41 smithi203 ceph-mon[35335]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:42.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:41 smithi189 ceph-mon[34851]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:42.243 INFO:journalctl@ceph.mds.g.smithi159.stdout:Sep 19 07:30:41 smithi159 systemd[1]: Starting Ceph mds.g for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:30:42.422 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:30:42.527 INFO:journalctl@ceph.mds.g.smithi159.stdout:Sep 19 07:30:42 smithi159 podman[71279]: 2024-09-19 07:30:42.147829071 +0000 UTC m=+0.027518295 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:30:42.527 INFO:journalctl@ceph.mds.g.smithi159.stdout:Sep 19 07:30:42 smithi159 podman[71279]: 2024-09-19 07:30:42.265987823 +0000 UTC m=+0.145677046 container create 9ef3eb9dc55758e9793aba8d7e4772d0f06117724a636dd65430fd75bfd07441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-g, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T07:30:42.527 INFO:journalctl@ceph.mds.g.smithi159.stdout:Sep 19 07:30:42 smithi159 podman[71279]: 2024-09-19 07:30:42.527220609 +0000 UTC m=+0.406909833 container init 9ef3eb9dc55758e9793aba8d7e4772d0f06117724a636dd65430fd75bfd07441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-g, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-19T07:30:42.819 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:42.887 INFO:journalctl@ceph.mds.g.smithi159.stdout:Sep 19 07:30:42 smithi159 podman[71279]: 2024-09-19 07:30:42.535719047 +0000 UTC m=+0.415408272 container start 9ef3eb9dc55758e9793aba8d7e4772d0f06117724a636dd65430fd75bfd07441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-g, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:30:42.887 INFO:journalctl@ceph.mds.g.smithi159.stdout:Sep 19 07:30:42 smithi159 ceph-mds[71316]: -- 172.21.15.159:0/474966409 <== mon.0 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56247139d860 con 0x56247137a800 2024-09-19T07:30:42.887 INFO:journalctl@ceph.mds.g.smithi159.stdout:Sep 19 07:30:42 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-g[71294]: starting mds.g at 2024-09-19T07:30:42.887 INFO:journalctl@ceph.mds.g.smithi159.stdout:Sep 19 07:30:42 smithi159 bash[71279]: 9ef3eb9dc55758e9793aba8d7e4772d0f06117724a636dd65430fd75bfd07441 2024-09-19T07:30:42.887 INFO:journalctl@ceph.mds.g.smithi159.stdout:Sep 19 07:30:42 smithi159 systemd[1]: Started Ceph mds.g for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:30:43.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.408+0000 7fa1fb200640 1 -- 172.21.15.159:0/220453424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1fc074f40 msgr2=0x7fa1fc10dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:43.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.408+0000 7fa1fb200640 1 --2- 172.21.15.159:0/220453424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1fc074f40 0x7fa1fc10dcc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009900 tx=0x7fa1e802a480 comp rx=0 tx=0).stop 2024-09-19T07:30:43.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.408+0000 7fa1fb200640 1 -- 172.21.15.159:0/220453424 shutdown_connections 2024-09-19T07:30:43.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.408+0000 7fa1fb200640 1 --2- 172.21.15.159:0/220453424 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc10e200 0x7fa1fc110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:43.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.408+0000 7fa1fb200640 1 --2- 172.21.15.159:0/220453424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1fc074f40 0x7fa1fc10dcc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:43.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.409+0000 7fa1fb200640 1 --2- 172.21.15.159:0/220453424 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1fc074620 0x7fa1fc074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:43.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.409+0000 7fa1fb200640 1 -- 172.21.15.159:0/220453424 >> 172.21.15.159:0/220453424 conn(0x7fa1fc0fd6e0 msgr2=0x7fa1fc0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:43.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.409+0000 7fa1fb200640 1 -- 172.21.15.159:0/220453424 shutdown_connections 2024-09-19T07:30:43.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.409+0000 7fa1fb200640 1 -- 172.21.15.159:0/220453424 wait complete. 2024-09-19T07:30:43.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.409+0000 7fa1fb200640 1 Processor -- start 2024-09-19T07:30:43.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.410+0000 7fa1fb200640 1 -- start start 2024-09-19T07:30:43.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.410+0000 7fa1fb200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc074620 0x7fa1fc19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:43.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.410+0000 7fa1fb200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1fc074f40 0x7fa1fc19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:43.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.410+0000 7fa1fb200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1fc10e200 0x7fa1fc19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:43.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.410+0000 7fa1fb200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc114000 con 0x7fa1fc074f40 2024-09-19T07:30:43.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.410+0000 7fa1fb200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc113e80 con 0x7fa1fc074620 2024-09-19T07:30:43.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.410+0000 7fa1fb200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc114180 con 0x7fa1fc10e200 2024-09-19T07:30:43.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.410+0000 7fa1f9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1fc074f40 0x7fa1fc19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:43.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.410+0000 7fa1f9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc074620 0x7fa1fc19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:43.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.410+0000 7fa1f9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1fc074f40 0x7fa1fc19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44160/0 (socket says 172.21.15.159:44160) 2024-09-19T07:30:43.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.410+0000 7fa1f9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc074620 0x7fa1fc19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:56914/0 (socket says 172.21.15.159:56914) 2024-09-19T07:30:43.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.410+0000 7fa1f9400640 1 -- 172.21.15.159:0/1438965174 learned_addr learned my addr 172.21.15.159:0/1438965174 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:30:43.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.410+0000 7fa1fa800640 1 --2- 172.21.15.159:0/1438965174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1fc10e200 0x7fa1fc19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:43.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.411+0000 7fa1f9e00640 1 -- 172.21.15.159:0/1438965174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1fc10e200 msgr2=0x7fa1fc19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:43.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.411+0000 7fa1f9e00640 1 --2- 172.21.15.159:0/1438965174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1fc10e200 0x7fa1fc19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:43.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.411+0000 7fa1f9e00640 1 -- 172.21.15.159:0/1438965174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1fc074f40 msgr2=0x7fa1fc19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:43.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.411+0000 7fa1f9e00640 1 --2- 172.21.15.159:0/1438965174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1fc074f40 0x7fa1fc19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:43.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.411+0000 7fa1f9e00640 1 -- 172.21.15.159:0/1438965174 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1fc19f460 con 0x7fa1fc074620 2024-09-19T07:30:43.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.411+0000 7fa1f9e00640 1 --2- 172.21.15.159:0/1438965174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc074620 0x7fa1fc19a4b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00e9f0 tx=0x7fa1ec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:43.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.411+0000 7fa1f2a00640 1 -- 172.21.15.159:0/1438965174 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec00ccc0 con 0x7fa1fc074620 2024-09-19T07:30:43.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.411+0000 7fa1fb200640 1 -- 172.21.15.159:0/1438965174 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1fc19f750 con 0x7fa1fc074620 2024-09-19T07:30:43.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.411+0000 7fa1f2a00640 1 -- 172.21.15.159:0/1438965174 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1ec005020 con 0x7fa1fc074620 2024-09-19T07:30:43.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.411+0000 7fa1f2a00640 1 -- 172.21.15.159:0/1438965174 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0049e0 con 0x7fa1fc074620 2024-09-19T07:30:43.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.411+0000 7fa1fb200640 1 -- 172.21.15.159:0/1438965174 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1fc1a7030 con 0x7fa1fc074620 2024-09-19T07:30:43.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.412+0000 7fa1fb200640 1 -- 172.21.15.159:0/1438965174 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1fc074a00 con 0x7fa1fc074620 2024-09-19T07:30:43.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.414+0000 7fa1f2a00640 1 -- 172.21.15.159:0/1438965174 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa1ec005560 con 0x7fa1fc074620 2024-09-19T07:30:43.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.416+0000 7fa1f2a00640 1 --2- 172.21.15.159:0/1438965174 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa1e00a3240 0x7fa1e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:43.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.416+0000 7fa1f2a00640 1 -- 172.21.15.159:0/1438965174 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa1ec0cf490 con 0x7fa1fc074620 2024-09-19T07:30:43.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.416+0000 7fa1f9400640 1 --2- 172.21.15.159:0/1438965174 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa1e00a3240 0x7fa1e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:43.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.416+0000 7fa1f9400640 1 --2- 172.21.15.159:0/1438965174 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa1e00a3240 0x7fa1e00a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80096c0 tx=0x7fa1e80046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:43.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.420+0000 7fa1f2a00640 1 -- 172.21.15.159:0/1438965174 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa1ec0051d0 con 0x7fa1fc074620 2024-09-19T07:30:43.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.648+0000 7fa1fb200640 1 -- 172.21.15.159:0/1438965174 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1fc02d7c0 con 0x7fa1fc074620 2024-09-19T07:30:43.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.649+0000 7fa1f2a00640 1 -- 172.21.15.159:0/1438965174 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8887 (secure 0 0 0) 0x7fa1ec09c5b0 con 0x7fa1fc074620 2024-09-19T07:30:43.650 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:30:43.650 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":11,"btime":"2024-09-19T07:30:42:738003+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3464449771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3464449771},{"type":"v1","addr":"172.21.15.159:6837","nonce":3464449771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2036264856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2036264856},{"type":"v1","addr":"172.21.15.159:6839","nonce":2036264856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1340279549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1340279549},{"type":"v1","addr":"172.21.15.189:6835","nonce":1340279549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/937970878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":937970878},{"type":"v1","addr":"172.21.15.203:6835","nonce":937970878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1053443810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1053443810},{"type":"v1","addr":"172.21.15.203:6837","nonce":1053443810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1030825585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1030825585},{"type":"v1","addr":"172.21.15.189:6837","nonce":1030825585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/3862781462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":3862781462},{"type":"v1","addr":"172.21.15.203:6839","nonce":3862781462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T07:30:43.650 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 11 2024-09-19T07:30:43.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.652+0000 7fa1fb200640 1 -- 172.21.15.159:0/1438965174 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa1e00a3240 msgr2=0x7fa1e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:43.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.652+0000 7fa1fb200640 1 --2- 172.21.15.159:0/1438965174 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa1e00a3240 0x7fa1e00a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80096c0 tx=0x7fa1e80046c0 comp rx=0 tx=0).stop 2024-09-19T07:30:43.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.652+0000 7fa1fb200640 1 -- 172.21.15.159:0/1438965174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc074620 msgr2=0x7fa1fc19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:43.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.652+0000 7fa1fb200640 1 --2- 172.21.15.159:0/1438965174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc074620 0x7fa1fc19a4b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00e9f0 tx=0x7fa1ec00eeb0 comp rx=0 tx=0).stop 2024-09-19T07:30:43.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.652+0000 7fa1fb200640 1 -- 172.21.15.159:0/1438965174 shutdown_connections 2024-09-19T07:30:43.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.652+0000 7fa1fb200640 1 --2- 172.21.15.159:0/1438965174 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa1e00a3240 0x7fa1e00a5700 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:43.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.652+0000 7fa1fb200640 1 --2- 172.21.15.159:0/1438965174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1fc10e200 0x7fa1fc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:43.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.652+0000 7fa1fb200640 1 --2- 172.21.15.159:0/1438965174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1fc074f40 0x7fa1fc19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:43.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.652+0000 7fa1fb200640 1 --2- 172.21.15.159:0/1438965174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc074620 0x7fa1fc19a4b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:43.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.652+0000 7fa1fb200640 1 -- 172.21.15.159:0/1438965174 >> 172.21.15.159:0/1438965174 conn(0x7fa1fc0fd6e0 msgr2=0x7fa1fc0fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:43.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.652+0000 7fa1fb200640 1 -- 172.21.15.159:0/1438965174 shutdown_connections 2024-09-19T07:30:43.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:43.652+0000 7fa1fb200640 1 -- 172.21.15.159:0/1438965174 wait complete. 2024-09-19T07:30:43.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:43 smithi159 ceph-mon[30472]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:43.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:43 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:43.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:43 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:43.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:43 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:43.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:43 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:43.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:43 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:43.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:43 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:43.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:43 smithi159 ceph-mon[30472]: Deploying daemon mds.a on smithi159 2024-09-19T07:30:43.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:43 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.159:6838/2036264856,v1:172.21.15.159:6839/2036264856] up:boot 2024-09-19T07:30:43.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:43 smithi159 ceph-mon[30472]: fsmap 10 up:standby 2024-09-19T07:30:43.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:43 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T07:30:43.888 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:43 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1438965174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:44.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:43 smithi203 ceph-mon[35335]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:44.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:43 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:44.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:43 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:44.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:43 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:44.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:43 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:44.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:43 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:44.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:43 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:44.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:43 smithi203 ceph-mon[35335]: Deploying daemon mds.a on smithi159 2024-09-19T07:30:44.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:43 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.159:6838/2036264856,v1:172.21.15.159:6839/2036264856] up:boot 2024-09-19T07:30:44.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:43 smithi203 ceph-mon[35335]: fsmap 10 up:standby 2024-09-19T07:30:44.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:43 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T07:30:44.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:43 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1438965174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:44.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:43 smithi189 ceph-mon[34851]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:44.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:43 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:44.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:43 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:44.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:43 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:44.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:43 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:44.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:43 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:44.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:43 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:44.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:43 smithi189 ceph-mon[34851]: Deploying daemon mds.a on smithi159 2024-09-19T07:30:44.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:43 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.159:6838/2036264856,v1:172.21.15.159:6839/2036264856] up:boot 2024-09-19T07:30:44.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:43 smithi189 ceph-mon[34851]: fsmap 10 up:standby 2024-09-19T07:30:44.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:43 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T07:30:44.057 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:43 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1438965174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:45.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:45 smithi159 ceph-mon[30472]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:46.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:45 smithi203 ceph-mon[35335]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:46.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:45 smithi189 ceph-mon[34851]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:46.619 INFO:journalctl@ceph.mds.a.smithi159.stdout:Sep 19 07:30:46 smithi159 systemd[1]: Starting Ceph mds.a for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:30:46.887 INFO:journalctl@ceph.mds.a.smithi159.stdout:Sep 19 07:30:46 smithi159 podman[71685]: 2024-09-19 07:30:46.523308993 +0000 UTC m=+0.027598767 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:30:46.887 INFO:journalctl@ceph.mds.a.smithi159.stdout:Sep 19 07:30:46 smithi159 podman[71685]: 2024-09-19 07:30:46.655173119 +0000 UTC m=+0.159462881 container create 202929e4eb2d165a1c5a64c0255cef401a47932b8794c791b09dde1e0370f4ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T07:30:47.095 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:30:47.387 INFO:journalctl@ceph.mds.a.smithi159.stdout:Sep 19 07:30:46 smithi159 podman[71685]: 2024-09-19 07:30:46.950372662 +0000 UTC m=+0.454662443 container init 202929e4eb2d165a1c5a64c0255cef401a47932b8794c791b09dde1e0370f4ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2) 2024-09-19T07:30:47.387 INFO:journalctl@ceph.mds.a.smithi159.stdout:Sep 19 07:30:46 smithi159 podman[71685]: 2024-09-19 07:30:46.956535458 +0000 UTC m=+0.460825221 container start 202929e4eb2d165a1c5a64c0255cef401a47932b8794c791b09dde1e0370f4ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T07:30:47.387 INFO:journalctl@ceph.mds.a.smithi159.stdout:Sep 19 07:30:46 smithi159 ceph-mds[71704]: -- 172.21.15.159:0/923348238 <== mon.0 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b9fb5d7c20 con 0x55b9fb5b5000 2024-09-19T07:30:47.387 INFO:journalctl@ceph.mds.a.smithi159.stdout:Sep 19 07:30:46 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-a[71700]: starting mds.a at 2024-09-19T07:30:47.387 INFO:journalctl@ceph.mds.a.smithi159.stdout:Sep 19 07:30:47 smithi159 bash[71685]: 202929e4eb2d165a1c5a64c0255cef401a47932b8794c791b09dde1e0370f4ea 2024-09-19T07:30:47.387 INFO:journalctl@ceph.mds.a.smithi159.stdout:Sep 19 07:30:47 smithi159 systemd[1]: Started Ceph mds.a for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:30:47.479 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:47.869 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:47 smithi189 ceph-mon[34851]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:47.870 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:47.870 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:47.870 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:47.870 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:47.870 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:47.870 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:47 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:47.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:47 smithi159 ceph-mon[30472]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:47.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:47.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:47.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:47.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:47.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:47.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:47 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:48.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:47 smithi203 ceph-mon[35335]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:48.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:48.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:48.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:48.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:30:48.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:30:48.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:47 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:48.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.065+0000 7fe0ac000640 1 -- 172.21.15.159:0/2457275701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0a4102790 msgr2=0x7fe0a41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:48.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.065+0000 7fe0ac000640 1 --2- 172.21.15.159:0/2457275701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0a4102790 0x7fe0a41067e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe090009900 tx=0x7fe09002a410 comp rx=0 tx=0).stop 2024-09-19T07:30:48.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.066+0000 7fe0ac000640 1 -- 172.21.15.159:0/2457275701 shutdown_connections 2024-09-19T07:30:48.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.066+0000 7fe0ac000640 1 --2- 172.21.15.159:0/2457275701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0a4106f10 0x7fe0a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:48.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.066+0000 7fe0ac000640 1 --2- 172.21.15.159:0/2457275701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0a4102790 0x7fe0a41067e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:48.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.066+0000 7fe0ac000640 1 --2- 172.21.15.159:0/2457275701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a4101de0 0x7fe0a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:48.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.066+0000 7fe0ac000640 1 -- 172.21.15.159:0/2457275701 >> 172.21.15.159:0/2457275701 conn(0x7fe0a40fd8c0 msgr2=0x7fe0a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:48.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.066+0000 7fe0ac000640 1 -- 172.21.15.159:0/2457275701 shutdown_connections 2024-09-19T07:30:48.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.066+0000 7fe0ac000640 1 -- 172.21.15.159:0/2457275701 wait complete. 2024-09-19T07:30:48.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.067+0000 7fe0ac000640 1 Processor -- start 2024-09-19T07:30:48.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.067+0000 7fe0ac000640 1 -- start start 2024-09-19T07:30:48.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.067+0000 7fe0ac000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0a4101de0 0x7fe0a419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:48.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.067+0000 7fe0ac000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a4102790 0x7fe0a419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:48.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.067+0000 7fe0ac000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0a4106f10 0x7fe0a419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:48.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.067+0000 7fe0ac000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a4111f80 con 0x7fe0a4106f10 2024-09-19T07:30:48.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.067+0000 7fe0ac000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a4111e00 con 0x7fe0a4101de0 2024-09-19T07:30:48.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.067+0000 7fe0ac000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a4112100 con 0x7fe0a4102790 2024-09-19T07:30:48.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.067+0000 7fe0a9600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0a4101de0 0x7fe0a419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:48.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.067+0000 7fe0aa000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0a4106f10 0x7fe0a419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:48.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.067+0000 7fe0a9600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0a4101de0 0x7fe0a419a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:56936/0 (socket says 172.21.15.159:56936) 2024-09-19T07:30:48.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.067+0000 7fe0a9600640 1 -- 172.21.15.159:0/413687235 learned_addr learned my addr 172.21.15.159:0/413687235 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:30:48.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.068+0000 7fe0a8c00640 1 --2- 172.21.15.159:0/413687235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a4102790 0x7fe0a419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:48.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.068+0000 7fe0a9600640 1 -- 172.21.15.159:0/413687235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a4102790 msgr2=0x7fe0a419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:48.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.068+0000 7fe0a9600640 1 --2- 172.21.15.159:0/413687235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a4102790 0x7fe0a419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:48.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.068+0000 7fe0a9600640 1 -- 172.21.15.159:0/413687235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0a4106f10 msgr2=0x7fe0a419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:48.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.068+0000 7fe0a9600640 1 --2- 172.21.15.159:0/413687235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0a4106f10 0x7fe0a419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:48.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.068+0000 7fe0a9600640 1 -- 172.21.15.159:0/413687235 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a419f630 con 0x7fe0a4101de0 2024-09-19T07:30:48.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.068+0000 7fe0aa000640 1 --2- 172.21.15.159:0/413687235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0a4106f10 0x7fe0a419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:30:48.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.068+0000 7fe0a9600640 1 --2- 172.21.15.159:0/413687235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0a4101de0 0x7fe0a419a680 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe09800eab0 tx=0x7fe09800ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:48.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.068+0000 7fe0a2000640 1 -- 172.21.15.159:0/413687235 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09800ce40 con 0x7fe0a4101de0 2024-09-19T07:30:48.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.068+0000 7fe0a2000640 1 -- 172.21.15.159:0/413687235 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe098011950 con 0x7fe0a4101de0 2024-09-19T07:30:48.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.068+0000 7fe0ac000640 1 -- 172.21.15.159:0/413687235 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a419f920 con 0x7fe0a4101de0 2024-09-19T07:30:48.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.068+0000 7fe0a2000640 1 -- 172.21.15.159:0/413687235 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098010ae0 con 0x7fe0a4101de0 2024-09-19T07:30:48.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.069+0000 7fe0ac000640 1 -- 172.21.15.159:0/413687235 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a41a7200 con 0x7fe0a4101de0 2024-09-19T07:30:48.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.070+0000 7fe0ac000640 1 -- 172.21.15.159:0/413687235 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0a41021c0 con 0x7fe0a4101de0 2024-09-19T07:30:48.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.071+0000 7fe0a2000640 1 -- 172.21.15.159:0/413687235 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fe098011b00 con 0x7fe0a4101de0 2024-09-19T07:30:48.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.073+0000 7fe0a2000640 1 --2- 172.21.15.159:0/413687235 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe08c0a3240 0x7fe08c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:48.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.073+0000 7fe0a2000640 1 -- 172.21.15.159:0/413687235 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe0980cfb50 con 0x7fe0a4101de0 2024-09-19T07:30:48.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.073+0000 7fe0a8c00640 1 --2- 172.21.15.159:0/413687235 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe08c0a3240 0x7fe08c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:48.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.074+0000 7fe0a8c00640 1 --2- 172.21.15.159:0/413687235 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe08c0a3240 0x7fe08c0a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe090000c00 tx=0x7fe090005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:48.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.076+0000 7fe0a2000640 1 -- 172.21.15.159:0/413687235 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe098016030 con 0x7fe0a4101de0 2024-09-19T07:30:48.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.286+0000 7fe0ac000640 1 -- 172.21.15.159:0/413687235 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0a419b9e0 con 0x7fe0a4101de0 2024-09-19T07:30:48.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.287+0000 7fe0a2000640 1 -- 172.21.15.159:0/413687235 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9716 (secure 0 0 0) 0x7fe0980bc070 con 0x7fe0a4101de0 2024-09-19T07:30:48.288 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:30:48.288 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":12,"btime":"2024-09-19T07:30:47:712020+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3464449771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3464449771},{"type":"v1","addr":"172.21.15.159:6837","nonce":3464449771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2036264856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2036264856},{"type":"v1","addr":"172.21.15.159:6839","nonce":2036264856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6841/2874566405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6840","nonce":2874566405},{"type":"v1","addr":"172.21.15.159:6841","nonce":2874566405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1340279549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1340279549},{"type":"v1","addr":"172.21.15.189:6835","nonce":1340279549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/937970878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":937970878},{"type":"v1","addr":"172.21.15.203:6835","nonce":937970878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1053443810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1053443810},{"type":"v1","addr":"172.21.15.203:6837","nonce":1053443810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1030825585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1030825585},{"type":"v1","addr":"172.21.15.189:6837","nonce":1030825585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/3862781462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":3862781462},{"type":"v1","addr":"172.21.15.203:6839","nonce":3862781462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T07:30:48.288 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 12 2024-09-19T07:30:48.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.290+0000 7fe0ac000640 1 -- 172.21.15.159:0/413687235 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe08c0a3240 msgr2=0x7fe08c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:48.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.290+0000 7fe0ac000640 1 --2- 172.21.15.159:0/413687235 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe08c0a3240 0x7fe08c0a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe090000c00 tx=0x7fe090005b50 comp rx=0 tx=0).stop 2024-09-19T07:30:48.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.290+0000 7fe0ac000640 1 -- 172.21.15.159:0/413687235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0a4101de0 msgr2=0x7fe0a419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:48.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.290+0000 7fe0ac000640 1 --2- 172.21.15.159:0/413687235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0a4101de0 0x7fe0a419a680 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe09800eab0 tx=0x7fe09800ef70 comp rx=0 tx=0).stop 2024-09-19T07:30:48.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.290+0000 7fe0ac000640 1 -- 172.21.15.159:0/413687235 shutdown_connections 2024-09-19T07:30:48.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.290+0000 7fe0ac000640 1 --2- 172.21.15.159:0/413687235 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe08c0a3240 0x7fe08c0a5700 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:48.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.290+0000 7fe0ac000640 1 --2- 172.21.15.159:0/413687235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0a4106f10 0x7fe0a419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:48.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.290+0000 7fe0ac000640 1 --2- 172.21.15.159:0/413687235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a4102790 0x7fe0a419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:48.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.290+0000 7fe0ac000640 1 --2- 172.21.15.159:0/413687235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0a4101de0 0x7fe0a419a680 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:48.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.290+0000 7fe0ac000640 1 -- 172.21.15.159:0/413687235 >> 172.21.15.159:0/413687235 conn(0x7fe0a40fd8c0 msgr2=0x7fe0a40ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:48.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.290+0000 7fe0ac000640 1 -- 172.21.15.159:0/413687235 shutdown_connections 2024-09-19T07:30:48.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:48.290+0000 7fe0ac000640 1 -- 172.21.15.159:0/413687235 wait complete. 2024-09-19T07:30:48.943 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:48 smithi189 ceph-mon[34851]: Deploying daemon mds.b on smithi189 2024-09-19T07:30:48.943 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:48 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.159:6840/2874566405,v1:172.21.15.159:6841/2874566405] up:boot 2024-09-19T07:30:48.943 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:48 smithi189 ceph-mon[34851]: fsmap 11 up:standby 2024-09-19T07:30:48.944 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:48 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T07:30:48.944 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:48 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/413687235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:49.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:48 smithi203 ceph-mon[35335]: Deploying daemon mds.b on smithi189 2024-09-19T07:30:49.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:48 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.159:6840/2874566405,v1:172.21.15.159:6841/2874566405] up:boot 2024-09-19T07:30:49.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:48 smithi203 ceph-mon[35335]: fsmap 11 up:standby 2024-09-19T07:30:49.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:48 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T07:30:49.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:48 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/413687235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:49.050 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:48 smithi159 ceph-mon[30472]: Deploying daemon mds.b on smithi189 2024-09-19T07:30:49.050 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:48 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.159:6840/2874566405,v1:172.21.15.159:6841/2874566405] up:boot 2024-09-19T07:30:49.051 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:48 smithi159 ceph-mon[30472]: fsmap 11 up:standby 2024-09-19T07:30:49.051 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:48 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T07:30:49.051 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:48 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/413687235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:49.306 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:30:49 smithi189 systemd[1]: Starting Ceph mds.b for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:30:49.604 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:30:49 smithi189 podman[64676]: 2024-09-19 07:30:49.243207646 +0000 UTC m=+0.016183369 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:30:49.604 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:30:49 smithi189 podman[64676]: 2024-09-19 07:30:49.343746582 +0000 UTC m=+0.116722334 container create 26d2b7f256a54bc9627232e16eb9f0bb643171be123cd282bd0b00b980cbd823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-19T07:30:49.605 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:30:49 smithi189 podman[64676]: 2024-09-19 07:30:49.604806898 +0000 UTC m=+0.377782607 container init 26d2b7f256a54bc9627232e16eb9f0bb643171be123cd282bd0b00b980cbd823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-b, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T07:30:50.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:49 smithi203 ceph-mon[35335]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:50.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:49 smithi189 ceph-mon[34851]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:50.056 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:30:49 smithi189 podman[64676]: 2024-09-19 07:30:49.607851085 +0000 UTC m=+0.380826812 container start 26d2b7f256a54bc9627232e16eb9f0bb643171be123cd282bd0b00b980cbd823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True) 2024-09-19T07:30:50.056 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:30:49 smithi189 ceph-mds[64695]: -- 172.21.15.189:0/3388176670 <== mon.2 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f2af011860 con 0x55f2aefee800 2024-09-19T07:30:50.056 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:30:49 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-b[64691]: starting mds.b at 2024-09-19T07:30:50.056 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:30:49 smithi189 bash[64676]: 26d2b7f256a54bc9627232e16eb9f0bb643171be123cd282bd0b00b980cbd823 2024-09-19T07:30:50.056 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:30:49 smithi189 systemd[1]: Started Ceph mds.b for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:30:50.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:49 smithi159 ceph-mon[30472]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:50.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:50 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] up:boot 2024-09-19T07:30:50.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:50 smithi189 ceph-mon[34851]: fsmap 12 up:standby 2024-09-19T07:30:50.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:50 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:30:50.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:50 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:50.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:50 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:50.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:50 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:50.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:50 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:50.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:50 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:50.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:50 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:50.807 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:50 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:51.029 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:50 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] up:boot 2024-09-19T07:30:51.029 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:50 smithi159 ceph-mon[30472]: fsmap 12 up:standby 2024-09-19T07:30:51.029 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:30:51.029 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:51.029 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:51.029 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:51.029 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:51.029 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:51.029 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:51.030 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:50 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:51.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:50 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] up:boot 2024-09-19T07:30:51.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:50 smithi203 ceph-mon[35335]: fsmap 12 up:standby 2024-09-19T07:30:51.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:50 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:30:51.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:50 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:51.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:50 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:51.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:50 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:51.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:50 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:51.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:50 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:51.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:50 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:51.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:50 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:51.090 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:30:51.505 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:51.994 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:51 smithi159 ceph-mon[30472]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:51.995 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:51 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:51.995 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:51 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:51.995 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:51 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:51.995 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:51 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:52.039 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:51 smithi203 ceph-mon[35335]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:52.039 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:51 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:52.039 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:51 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:52.039 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:51 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:52.039 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:51 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:52.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:51 smithi189 ceph-mon[34851]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:52.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:51 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:52.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:51 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:52.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:51 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:52.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:51 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:52.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.102+0000 7fafa8c00640 1 -- 172.21.15.159:0/4024762031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa41079c0 msgr2=0x7fafa4107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:52.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.102+0000 7fafa8c00640 1 --2- 172.21.15.159:0/4024762031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa41079c0 0x7fafa4107da0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faf94009ca0 tx=0x7faf9402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:30:52.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.103+0000 7fafa8c00640 1 -- 172.21.15.159:0/4024762031 shutdown_connections 2024-09-19T07:30:52.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.103+0000 7fafa8c00640 1 --2- 172.21.15.159:0/4024762031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafa4102360 0x7fafa410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:52.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.103+0000 7fafa8c00640 1 --2- 172.21.15.159:0/4024762031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafa41019c0 0x7fafa4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:52.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.103+0000 7fafa8c00640 1 --2- 172.21.15.159:0/4024762031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa41079c0 0x7fafa4107da0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:52.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.103+0000 7fafa8c00640 1 -- 172.21.15.159:0/4024762031 >> 172.21.15.159:0/4024762031 conn(0x7fafa40fd740 msgr2=0x7fafa40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:52.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.103+0000 7fafa8c00640 1 -- 172.21.15.159:0/4024762031 shutdown_connections 2024-09-19T07:30:52.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.103+0000 7fafa8c00640 1 -- 172.21.15.159:0/4024762031 wait complete. 2024-09-19T07:30:52.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.104+0000 7fafa8c00640 1 Processor -- start 2024-09-19T07:30:52.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.104+0000 7fafa8c00640 1 -- start start 2024-09-19T07:30:52.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.104+0000 7fafa8c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa41019c0 0x7fafa419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:52.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.104+0000 7fafa8c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafa4102360 0x7fafa419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:52.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.104+0000 7fafa8c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafa41079c0 0x7fafa419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:52.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.104+0000 7fafa8c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fafa4111dd0 con 0x7fafa41079c0 2024-09-19T07:30:52.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.104+0000 7fafa8c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fafa4111c50 con 0x7fafa41019c0 2024-09-19T07:30:52.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.104+0000 7fafa8c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fafa4111f50 con 0x7fafa4102360 2024-09-19T07:30:52.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.104+0000 7fafa3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa41019c0 0x7fafa419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:52.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.104+0000 7fafa3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa41019c0 0x7fafa419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:51192/0 (socket says 172.21.15.159:51192) 2024-09-19T07:30:52.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.104+0000 7fafa3400640 1 -- 172.21.15.159:0/316214258 learned_addr learned my addr 172.21.15.159:0/316214258 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:30:52.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.105+0000 7fafa2a00640 1 --2- 172.21.15.159:0/316214258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafa4102360 0x7fafa419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:52.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.105+0000 7fafa3e00640 1 --2- 172.21.15.159:0/316214258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafa41079c0 0x7fafa419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:52.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.105+0000 7fafa3400640 1 -- 172.21.15.159:0/316214258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafa4102360 msgr2=0x7fafa419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:52.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.105+0000 7fafa3400640 1 --2- 172.21.15.159:0/316214258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafa4102360 0x7fafa419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:52.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.105+0000 7fafa3400640 1 -- 172.21.15.159:0/316214258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafa41079c0 msgr2=0x7fafa419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:52.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.105+0000 7fafa3400640 1 --2- 172.21.15.159:0/316214258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafa41079c0 0x7fafa419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:52.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.105+0000 7fafa3400640 1 -- 172.21.15.159:0/316214258 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa419f430 con 0x7fafa41019c0 2024-09-19T07:30:52.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.105+0000 7fafa3e00640 1 --2- 172.21.15.159:0/316214258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafa41079c0 0x7fafa419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:30:52.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.105+0000 7fafa3400640 1 --2- 172.21.15.159:0/316214258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa41019c0 0x7fafa419a480 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faf94005e30 tx=0x7faf94003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:52.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.105+0000 7faf9be00640 1 -- 172.21.15.159:0/316214258 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf94031650 con 0x7fafa41019c0 2024-09-19T07:30:52.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.105+0000 7faf9be00640 1 -- 172.21.15.159:0/316214258 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf940317f0 con 0x7fafa41019c0 2024-09-19T07:30:52.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.105+0000 7fafa8c00640 1 -- 172.21.15.159:0/316214258 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa419f6c0 con 0x7fafa41019c0 2024-09-19T07:30:52.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.106+0000 7faf9be00640 1 -- 172.21.15.159:0/316214258 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9402faa0 con 0x7fafa41019c0 2024-09-19T07:30:52.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.106+0000 7fafa8c00640 1 -- 172.21.15.159:0/316214258 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa41a7000 con 0x7fafa41019c0 2024-09-19T07:30:52.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.107+0000 7fafa8c00640 1 -- 172.21.15.159:0/316214258 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa4101e20 con 0x7fafa41019c0 2024-09-19T07:30:52.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.108+0000 7faf9be00640 1 -- 172.21.15.159:0/316214258 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7faf9402fc40 con 0x7fafa41019c0 2024-09-19T07:30:52.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.110+0000 7faf9be00640 1 --2- 172.21.15.159:0/316214258 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7faf880a32c0 0x7faf880a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:52.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.110+0000 7faf9be00640 1 -- 172.21.15.159:0/316214258 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7faf940f0840 con 0x7fafa41019c0 2024-09-19T07:30:52.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.110+0000 7fafa2a00640 1 --2- 172.21.15.159:0/316214258 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7faf880a32c0 0x7faf880a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:52.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.111+0000 7fafa2a00640 1 --2- 172.21.15.159:0/316214258 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7faf880a32c0 0x7faf880a5780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fafa410ec90 tx=0x7faf8c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:52.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.113+0000 7faf9be00640 1 -- 172.21.15.159:0/316214258 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faf940bd960 con 0x7fafa41019c0 2024-09-19T07:30:52.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.354+0000 7fafa8c00640 1 -- 172.21.15.159:0/316214258 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafa41a71d0 con 0x7fafa41019c0 2024-09-19T07:30:52.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.355+0000 7faf9be00640 1 -- 172.21.15.159:0/316214258 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10545 (secure 0 0 0) 0x7faf940bd260 con 0x7fafa41019c0 2024-09-19T07:30:52.417 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:30:52.417 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":13,"btime":"2024-09-19T07:30:49:728895+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3464449771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3464449771},{"type":"v1","addr":"172.21.15.159:6837","nonce":3464449771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2036264856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2036264856},{"type":"v1","addr":"172.21.15.159:6839","nonce":2036264856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6841/2874566405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6840","nonce":2874566405},{"type":"v1","addr":"172.21.15.159:6841","nonce":2874566405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1340279549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1340279549},{"type":"v1","addr":"172.21.15.189:6835","nonce":1340279549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/937970878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":937970878},{"type":"v1","addr":"172.21.15.203:6835","nonce":937970878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1053443810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1053443810},{"type":"v1","addr":"172.21.15.203:6837","nonce":1053443810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1030825585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1030825585},{"type":"v1","addr":"172.21.15.189:6837","nonce":1030825585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/3862781462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":3862781462},{"type":"v1","addr":"172.21.15.203:6839","nonce":3862781462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24542,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/2361818663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2361818663},{"type":"v1","addr":"172.21.15.189:6839","nonce":2361818663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-19T07:30:52.418 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 13 2024-09-19T07:30:52.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.357+0000 7fafa8c00640 1 -- 172.21.15.159:0/316214258 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7faf880a32c0 msgr2=0x7faf880a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:52.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.357+0000 7fafa8c00640 1 --2- 172.21.15.159:0/316214258 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7faf880a32c0 0x7faf880a5780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fafa410ec90 tx=0x7faf8c005f90 comp rx=0 tx=0).stop 2024-09-19T07:30:52.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.357+0000 7fafa8c00640 1 -- 172.21.15.159:0/316214258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa41019c0 msgr2=0x7fafa419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:52.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.357+0000 7fafa8c00640 1 --2- 172.21.15.159:0/316214258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa41019c0 0x7fafa419a480 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faf94005e30 tx=0x7faf94003f90 comp rx=0 tx=0).stop 2024-09-19T07:30:52.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.357+0000 7fafa8c00640 1 -- 172.21.15.159:0/316214258 shutdown_connections 2024-09-19T07:30:52.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.357+0000 7fafa8c00640 1 --2- 172.21.15.159:0/316214258 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7faf880a32c0 0x7faf880a5780 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:52.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.357+0000 7fafa8c00640 1 --2- 172.21.15.159:0/316214258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafa41079c0 0x7fafa419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:52.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.357+0000 7fafa8c00640 1 --2- 172.21.15.159:0/316214258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafa4102360 0x7fafa419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:52.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.357+0000 7fafa8c00640 1 --2- 172.21.15.159:0/316214258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa41019c0 0x7fafa419a480 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:52.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.357+0000 7fafa8c00640 1 -- 172.21.15.159:0/316214258 >> 172.21.15.159:0/316214258 conn(0x7fafa40fd740 msgr2=0x7fafa40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:52.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.358+0000 7fafa8c00640 1 -- 172.21.15.159:0/316214258 shutdown_connections 2024-09-19T07:30:52.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:52.359+0000 7fafa8c00640 1 -- 172.21.15.159:0/316214258 wait complete. 2024-09-19T07:30:53.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:52 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/316214258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:53.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:52 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/316214258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:53.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:52 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/316214258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:30:53.311 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-19T07:30:53.705 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:54.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:53 smithi203 ceph-mon[35335]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:54.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:53 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:54.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:53 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:54.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:53 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:54.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:53 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:54.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:53 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:54.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:53 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:54.038 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:53 smithi159 ceph-mon[30472]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:54.038 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:53 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:54.038 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:53 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:54.038 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:53 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:54.038 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:53 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:54.038 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:53 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:54.038 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:53 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:54.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:53 smithi189 ceph-mon[34851]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:54.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:53 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:54.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:53 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:54.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:53 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:54.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:53 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:54.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:53 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:54.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:53 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:54.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.328+0000 7f15b4600640 1 -- 172.21.15.159:0/3109347747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15ac106f10 msgr2=0x7f15ac10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:54.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.328+0000 7f15b4600640 1 --2- 172.21.15.159:0/3109347747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15ac106f10 0x7f15ac10ea90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009ca0 tx=0x7f15a402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:30:54.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.328+0000 7f15b4600640 1 -- 172.21.15.159:0/3109347747 shutdown_connections 2024-09-19T07:30:54.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.328+0000 7f15b4600640 1 --2- 172.21.15.159:0/3109347747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15ac106f10 0x7f15ac10ea90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:54.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.328+0000 7f15b4600640 1 --2- 172.21.15.159:0/3109347747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15ac102790 0x7f15ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:54.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.328+0000 7f15b4600640 1 --2- 172.21.15.159:0/3109347747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15ac101de0 0x7f15ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:54.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.328+0000 7f15b4600640 1 -- 172.21.15.159:0/3109347747 >> 172.21.15.159:0/3109347747 conn(0x7f15ac0fd8c0 msgr2=0x7f15ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:54.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.328+0000 7f15b4600640 1 -- 172.21.15.159:0/3109347747 shutdown_connections 2024-09-19T07:30:54.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.328+0000 7f15b4600640 1 -- 172.21.15.159:0/3109347747 wait complete. 2024-09-19T07:30:54.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.329+0000 7f15b4600640 1 Processor -- start 2024-09-19T07:30:54.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.329+0000 7f15b4600640 1 -- start start 2024-09-19T07:30:54.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.329+0000 7f15b4600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15ac101de0 0x7f15ac19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:54.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.329+0000 7f15b4600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15ac102790 0x7f15ac19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:54.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.330+0000 7f15b4600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15ac106f10 0x7f15ac19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:54.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.330+0000 7f15b4600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac111f70 con 0x7f15ac106f10 2024-09-19T07:30:54.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.330+0000 7f15b4600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac111df0 con 0x7f15ac101de0 2024-09-19T07:30:54.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.330+0000 7f15b4600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac1120f0 con 0x7f15ac102790 2024-09-19T07:30:54.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.330+0000 7f15b1c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15ac101de0 0x7f15ac19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:54.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.330+0000 7f15b1c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15ac101de0 0x7f15ac19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:51206/0 (socket says 172.21.15.159:51206) 2024-09-19T07:30:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.330+0000 7f15b1c00640 1 -- 172.21.15.159:0/1834500428 learned_addr learned my addr 172.21.15.159:0/1834500428 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:30:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.330+0000 7f15b2600640 1 --2- 172.21.15.159:0/1834500428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15ac106f10 0x7f15ac19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.330+0000 7f15b1200640 1 --2- 172.21.15.159:0/1834500428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15ac102790 0x7f15ac19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.330+0000 7f15b1c00640 1 -- 172.21.15.159:0/1834500428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15ac102790 msgr2=0x7f15ac19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.330+0000 7f15b1c00640 1 --2- 172.21.15.159:0/1834500428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15ac102790 0x7f15ac19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.330+0000 7f15b1c00640 1 -- 172.21.15.159:0/1834500428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15ac106f10 msgr2=0x7f15ac19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.330+0000 7f15b1c00640 1 --2- 172.21.15.159:0/1834500428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15ac106f10 0x7f15ac19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.330+0000 7f15b1c00640 1 -- 172.21.15.159:0/1834500428 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ac19f590 con 0x7f15ac101de0 2024-09-19T07:30:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.330+0000 7f15b1c00640 1 --2- 172.21.15.159:0/1834500428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15ac101de0 0x7f15ac19a5e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f15a000e9f0 tx=0x7f15a000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.331+0000 7f15aa000640 1 -- 172.21.15.159:0/1834500428 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a000cd80 con 0x7f15ac101de0 2024-09-19T07:30:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.331+0000 7f15aa000640 1 -- 172.21.15.159:0/1834500428 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15a0005500 con 0x7f15ac101de0 2024-09-19T07:30:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.331+0000 7f15b4600640 1 -- 172.21.15.159:0/1834500428 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ac19f880 con 0x7f15ac101de0 2024-09-19T07:30:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.331+0000 7f15aa000640 1 -- 172.21.15.159:0/1834500428 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0010690 con 0x7f15ac101de0 2024-09-19T07:30:54.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.331+0000 7f15b4600640 1 -- 172.21.15.159:0/1834500428 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ac1a7160 con 0x7f15ac101de0 2024-09-19T07:30:54.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.332+0000 7f15b4600640 1 -- 172.21.15.159:0/1834500428 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ac1021c0 con 0x7f15ac101de0 2024-09-19T07:30:54.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.333+0000 7f15aa000640 1 -- 172.21.15.159:0/1834500428 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f15a00046c0 con 0x7f15ac101de0 2024-09-19T07:30:54.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.335+0000 7f15aa000640 1 --2- 172.21.15.159:0/1834500428 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f15940a32c0 0x7f15940a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:54.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.335+0000 7f15aa000640 1 -- 172.21.15.159:0/1834500428 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f15a00cf890 con 0x7f15ac101de0 2024-09-19T07:30:54.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.335+0000 7f15b1200640 1 --2- 172.21.15.159:0/1834500428 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f15940a32c0 0x7f15940a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:54.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.336+0000 7f15b1200640 1 --2- 172.21.15.159:0/1834500428 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f15940a32c0 0x7f15940a5780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f15ac10ef40 tx=0x7f1598005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:54.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.338+0000 7f15aa000640 1 -- 172.21.15.159:0/1834500428 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f15a009c9b0 con 0x7f15ac101de0 2024-09-19T07:30:54.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.528+0000 7f15b4600640 1 -- 172.21.15.159:0/1834500428 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f15ac19b940 con 0x7f15ac101de0 2024-09-19T07:30:54.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.529+0000 7f15aa000640 1 -- 172.21.15.159:0/1834500428 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f15a0096e20 con 0x7f15ac101de0 2024-09-19T07:30:54.535 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:30:54.535 INFO:teuthology.orchestra.run.smithi159.stdout:[ 2024-09-19T07:30:54.535 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:30:54.536 INFO:teuthology.orchestra.run.smithi159.stdout: "poolnum": 1, 2024-09-19T07:30:54.536 INFO:teuthology.orchestra.run.smithi159.stdout: "poolname": ".mgr" 2024-09-19T07:30:54.536 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:30:54.536 INFO:teuthology.orchestra.run.smithi159.stdout:] 2024-09-19T07:30:54.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.531+0000 7f15b4600640 1 -- 172.21.15.159:0/1834500428 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f15940a32c0 msgr2=0x7f15940a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:54.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.531+0000 7f15b4600640 1 --2- 172.21.15.159:0/1834500428 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f15940a32c0 0x7f15940a5780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f15ac10ef40 tx=0x7f1598005f90 comp rx=0 tx=0).stop 2024-09-19T07:30:54.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.531+0000 7f15b4600640 1 -- 172.21.15.159:0/1834500428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15ac101de0 msgr2=0x7f15ac19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:54.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.531+0000 7f15b4600640 1 --2- 172.21.15.159:0/1834500428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15ac101de0 0x7f15ac19a5e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f15a000e9f0 tx=0x7f15a000edf0 comp rx=0 tx=0).stop 2024-09-19T07:30:54.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.531+0000 7f15b4600640 1 -- 172.21.15.159:0/1834500428 shutdown_connections 2024-09-19T07:30:54.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.531+0000 7f15b4600640 1 --2- 172.21.15.159:0/1834500428 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f15940a32c0 0x7f15940a5780 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:54.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.531+0000 7f15b4600640 1 --2- 172.21.15.159:0/1834500428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15ac106f10 0x7f15ac19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:54.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.531+0000 7f15b4600640 1 --2- 172.21.15.159:0/1834500428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15ac102790 0x7f15ac19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:54.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.531+0000 7f15b4600640 1 --2- 172.21.15.159:0/1834500428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15ac101de0 0x7f15ac19a5e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:54.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.531+0000 7f15b4600640 1 -- 172.21.15.159:0/1834500428 >> 172.21.15.159:0/1834500428 conn(0x7f15ac0fd8c0 msgr2=0x7f15ac0ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:54.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.531+0000 7f15b4600640 1 -- 172.21.15.159:0/1834500428 shutdown_connections 2024-09-19T07:30:54.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:54.531+0000 7f15b4600640 1 -- 172.21.15.159:0/1834500428 wait complete. 2024-09-19T07:30:55.031 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:54 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1834500428' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T07:30:55.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:54 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1834500428' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T07:30:55.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:54 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1834500428' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T07:30:56.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:55 smithi203 ceph-mon[35335]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:56.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:55 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:56.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:55 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:56.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:55 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:56.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:55 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:56.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:55 smithi189 ceph-mon[34851]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:56.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:55 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:56.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:55 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:56.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:55 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:56.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:55 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:56.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:55 smithi159 ceph-mon[30472]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:56.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:55 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:56.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:55 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:56.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:55 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:56.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:55 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:56.281 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T07:30:56.282 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T07:30:56.679 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:57.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.314+0000 7f2797e00640 1 -- 172.21.15.159:0/4160585621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2790104010 msgr2=0x7f27901043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:57.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.314+0000 7f2797e00640 1 --2- 172.21.15.159:0/4160585621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2790104010 0x7f27901043f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2784009900 tx=0x7f278402a3f0 comp rx=0 tx=0).stop 2024-09-19T07:30:57.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.314+0000 7f2797e00640 1 -- 172.21.15.159:0/4160585621 shutdown_connections 2024-09-19T07:30:57.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.314+0000 7f2797e00640 1 --2- 172.21.15.159:0/4160585621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2790109140 0x7f2790110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:57.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.314+0000 7f2797e00640 1 --2- 172.21.15.159:0/4160585621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27901049c0 0x7f2790108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:57.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.314+0000 7f2797e00640 1 --2- 172.21.15.159:0/4160585621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2790104010 0x7f27901043f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:57.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.314+0000 7f2797e00640 1 -- 172.21.15.159:0/4160585621 >> 172.21.15.159:0/4160585621 conn(0x7f27900ffa50 msgr2=0x7f2790101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:57.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.314+0000 7f2797e00640 1 -- 172.21.15.159:0/4160585621 shutdown_connections 2024-09-19T07:30:57.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.314+0000 7f2797e00640 1 -- 172.21.15.159:0/4160585621 wait complete. 2024-09-19T07:30:57.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.315+0000 7f2797e00640 1 Processor -- start 2024-09-19T07:30:57.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.315+0000 7f2797e00640 1 -- start start 2024-09-19T07:30:57.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.315+0000 7f2797e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2790104010 0x7f279019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:57.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.315+0000 7f2797e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27901049c0 0x7f279019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:57.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.315+0000 7f2797e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2790109140 0x7f279019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:57.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.315+0000 7f2797e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27901141b0 con 0x7f27901049c0 2024-09-19T07:30:57.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.315+0000 7f2797e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2790114030 con 0x7f2790104010 2024-09-19T07:30:57.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.315+0000 7f2797e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2790114330 con 0x7f2790109140 2024-09-19T07:30:57.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.316+0000 7f2794a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27901049c0 0x7f279019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:57.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.316+0000 7f2794a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27901049c0 0x7f279019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38284/0 (socket says 172.21.15.159:38284) 2024-09-19T07:30:57.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.316+0000 7f2794a00640 1 -- 172.21.15.159:0/536792862 learned_addr learned my addr 172.21.15.159:0/536792862 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:30:57.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.316+0000 7f2795400640 1 --2- 172.21.15.159:0/536792862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2790104010 0x7f279019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:57.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.316+0000 7f2795e00640 1 --2- 172.21.15.159:0/536792862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2790109140 0x7f279019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:57.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.316+0000 7f2794a00640 1 -- 172.21.15.159:0/536792862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2790109140 msgr2=0x7f279019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:57.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.316+0000 7f2794a00640 1 --2- 172.21.15.159:0/536792862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2790109140 0x7f279019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:57.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.316+0000 7f2794a00640 1 -- 172.21.15.159:0/536792862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2790104010 msgr2=0x7f279019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:57.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.316+0000 7f2794a00640 1 --2- 172.21.15.159:0/536792862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2790104010 0x7f279019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:57.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.316+0000 7f2794a00640 1 -- 172.21.15.159:0/536792862 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279019f600 con 0x7f27901049c0 2024-09-19T07:30:57.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.316+0000 7f2794a00640 1 --2- 172.21.15.159:0/536792862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27901049c0 0x7f279019ab90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f277c00e9f0 tx=0x7f277c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:57.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.317+0000 7f278e000640 1 -- 172.21.15.159:0/536792862 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c00cd80 con 0x7f27901049c0 2024-09-19T07:30:57.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.317+0000 7f278e000640 1 -- 172.21.15.159:0/536792862 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f277c005500 con 0x7f27901049c0 2024-09-19T07:30:57.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.317+0000 7f278e000640 1 -- 172.21.15.159:0/536792862 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c010690 con 0x7f27901049c0 2024-09-19T07:30:57.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.317+0000 7f2797e00640 1 -- 172.21.15.159:0/536792862 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279019f8f0 con 0x7f27901049c0 2024-09-19T07:30:57.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.317+0000 7f2797e00640 1 -- 172.21.15.159:0/536792862 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27901a71d0 con 0x7f27901049c0 2024-09-19T07:30:57.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.318+0000 7f2797e00640 1 -- 172.21.15.159:0/536792862 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27901043f0 con 0x7f27901049c0 2024-09-19T07:30:57.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.320+0000 7f278e000640 1 -- 172.21.15.159:0/536792862 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f277c0046c0 con 0x7f27901049c0 2024-09-19T07:30:57.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.322+0000 7f278e000640 1 --2- 172.21.15.159:0/536792862 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f27780a32c0 0x7f27780a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:57.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.322+0000 7f278e000640 1 -- 172.21.15.159:0/536792862 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f277c0cf890 con 0x7f27901049c0 2024-09-19T07:30:57.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.322+0000 7f2795400640 1 --2- 172.21.15.159:0/536792862 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f27780a32c0 0x7f27780a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:57.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.323+0000 7f2795400640 1 --2- 172.21.15.159:0/536792862 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f27780a32c0 0x7f27780a5780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2784000c00 tx=0x7f2784005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:57.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.326+0000 7f278e000640 1 -- 172.21.15.159:0/536792862 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f277c09c9b0 con 0x7f27901049c0 2024-09-19T07:30:57.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:57.516+0000 7f2797e00640 1 -- 172.21.15.159:0/536792862 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f279002d7c0 con 0x7f27901049c0 2024-09-19T07:30:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:58 smithi203 ceph-mon[35335]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:58 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:58 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:58 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:58 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:58 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:58.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:58 smithi189 ceph-mon[34851]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:58.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:58 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:58.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:58 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:58.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:58 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:58.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:58 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:58.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:58 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:58.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:58 smithi159 ceph-mon[30472]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:58.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:58 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:58.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:58 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:58.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:58 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:58.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:58 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:58.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:58 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:30:58.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:58.391+0000 7f278e000640 1 -- 172.21.15.159:0/536792862 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f277c096e20 con 0x7f27901049c0 2024-09-19T07:30:58.391 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' created 2024-09-19T07:30:58.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:58.393+0000 7f2797e00640 1 -- 172.21.15.159:0/536792862 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f27780a32c0 msgr2=0x7f27780a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:58.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:58.393+0000 7f2797e00640 1 --2- 172.21.15.159:0/536792862 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f27780a32c0 0x7f27780a5780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2784000c00 tx=0x7f2784005b50 comp rx=0 tx=0).stop 2024-09-19T07:30:58.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:58.393+0000 7f2797e00640 1 -- 172.21.15.159:0/536792862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27901049c0 msgr2=0x7f279019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:58.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:58.393+0000 7f2797e00640 1 --2- 172.21.15.159:0/536792862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27901049c0 0x7f279019ab90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f277c00e9f0 tx=0x7f277c00edf0 comp rx=0 tx=0).stop 2024-09-19T07:30:58.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:58.393+0000 7f2797e00640 1 -- 172.21.15.159:0/536792862 shutdown_connections 2024-09-19T07:30:58.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:58.393+0000 7f2797e00640 1 --2- 172.21.15.159:0/536792862 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f27780a32c0 0x7f27780a5780 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:58.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:58.393+0000 7f2797e00640 1 --2- 172.21.15.159:0/536792862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2790109140 0x7f279019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:58.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:58.393+0000 7f2797e00640 1 --2- 172.21.15.159:0/536792862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27901049c0 0x7f279019ab90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:58.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:58.393+0000 7f2797e00640 1 --2- 172.21.15.159:0/536792862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2790104010 0x7f279019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:58.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:58.393+0000 7f2797e00640 1 -- 172.21.15.159:0/536792862 >> 172.21.15.159:0/536792862 conn(0x7f27900ffa50 msgr2=0x7f2790100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:58.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:58.394+0000 7f2797e00640 1 -- 172.21.15.159:0/536792862 shutdown_connections 2024-09-19T07:30:58.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:30:58.394+0000 7f2797e00640 1 -- 172.21.15.159:0/536792862 wait complete. 2024-09-19T07:30:59.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:59 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/536792862' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T07:30:59.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:59 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/536792862' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T07:30:59.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:59 smithi159 ceph-mon[30472]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T07:30:59.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:30:59 smithi159 ceph-mon[30472]: pgmap v297: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:59.265 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-19T07:30:59.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:59 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/536792862' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T07:30:59.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:59 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/536792862' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T07:30:59.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:59 smithi203 ceph-mon[35335]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T07:30:59.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:30:59 smithi203 ceph-mon[35335]: pgmap v297: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:59.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:59 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/536792862' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T07:30:59.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:59 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/536792862' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T07:30:59.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:59 smithi189 ceph-mon[34851]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T07:30:59.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:30:59 smithi189 ceph-mon[34851]: pgmap v297: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:30:59.662 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:00.057 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:00 smithi159 ceph-mon[30472]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T07:31:00.057 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:00 smithi159 ceph-mon[30472]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T07:31:00.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:00 smithi203 ceph-mon[35335]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T07:31:00.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:00 smithi203 ceph-mon[35335]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T07:31:00.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:00 smithi189 ceph-mon[34851]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T07:31:00.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:00 smithi189 ceph-mon[34851]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T07:31:00.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.349+0000 7f6a93200640 1 -- 172.21.15.159:0/262209509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a8c1049c0 msgr2=0x7f6a8c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:00.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.349+0000 7f6a93200640 1 --2- 172.21.15.159:0/262209509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a8c1049c0 0x7f6a8c108a10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009860 tx=0x7f6a7402a340 comp rx=0 tx=0).stop 2024-09-19T07:31:00.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.350+0000 7f6a93200640 1 -- 172.21.15.159:0/262209509 shutdown_connections 2024-09-19T07:31:00.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.350+0000 7f6a93200640 1 --2- 172.21.15.159:0/262209509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a8c109140 0x7f6a8c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:00.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.350+0000 7f6a93200640 1 --2- 172.21.15.159:0/262209509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a8c1049c0 0x7f6a8c108a10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:00.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.350+0000 7f6a93200640 1 --2- 172.21.15.159:0/262209509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a8c104010 0x7f6a8c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:00.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.350+0000 7f6a93200640 1 -- 172.21.15.159:0/262209509 >> 172.21.15.159:0/262209509 conn(0x7f6a8c0ffa50 msgr2=0x7f6a8c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:00.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.350+0000 7f6a93200640 1 -- 172.21.15.159:0/262209509 shutdown_connections 2024-09-19T07:31:00.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.350+0000 7f6a93200640 1 -- 172.21.15.159:0/262209509 wait complete. 2024-09-19T07:31:00.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.351+0000 7f6a93200640 1 Processor -- start 2024-09-19T07:31:00.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.351+0000 7f6a93200640 1 -- start start 2024-09-19T07:31:00.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.351+0000 7f6a93200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a8c104010 0x7f6a8c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:00.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.351+0000 7f6a93200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a8c1049c0 0x7f6a8c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:00.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.351+0000 7f6a93200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a8c109140 0x7f6a8c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:00.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.351+0000 7f6a93200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c114140 con 0x7f6a8c1049c0 2024-09-19T07:31:00.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.351+0000 7f6a93200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c113fc0 con 0x7f6a8c109140 2024-09-19T07:31:00.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.351+0000 7f6a93200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c1142c0 con 0x7f6a8c104010 2024-09-19T07:31:00.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.351+0000 7f6a83400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a8c1049c0 0x7f6a8c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:00.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.351+0000 7f6a83400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a8c1049c0 0x7f6a8c19aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38294/0 (socket says 172.21.15.159:38294) 2024-09-19T07:31:00.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.351+0000 7f6a83400640 1 -- 172.21.15.159:0/4192042614 learned_addr learned my addr 172.21.15.159:0/4192042614 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:00.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.351+0000 7f6a8be00640 1 --2- 172.21.15.159:0/4192042614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a8c104010 0x7f6a8c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:00.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.351+0000 7f6a91200640 1 --2- 172.21.15.159:0/4192042614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a8c109140 0x7f6a8c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:00.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.352+0000 7f6a8be00640 1 -- 172.21.15.159:0/4192042614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a8c109140 msgr2=0x7f6a8c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:00.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.352+0000 7f6a8be00640 1 --2- 172.21.15.159:0/4192042614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a8c109140 0x7f6a8c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:00.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.352+0000 7f6a8be00640 1 -- 172.21.15.159:0/4192042614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a8c1049c0 msgr2=0x7f6a8c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:00.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.352+0000 7f6a8be00640 1 --2- 172.21.15.159:0/4192042614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a8c1049c0 0x7f6a8c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:00.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.352+0000 7f6a8be00640 1 -- 172.21.15.159:0/4192042614 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8c19f610 con 0x7f6a8c104010 2024-09-19T07:31:00.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.352+0000 7f6a83400640 1 --2- 172.21.15.159:0/4192042614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a8c1049c0 0x7f6a8c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:31:00.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.352+0000 7f6a8be00640 1 --2- 172.21.15.159:0/4192042614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a8c104010 0x7f6a8c19a660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009fd0 tx=0x7f6a7c00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:00.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.352+0000 7f6a89600640 1 -- 172.21.15.159:0/4192042614 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c003970 con 0x7f6a8c104010 2024-09-19T07:31:00.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.352+0000 7f6a93200640 1 -- 172.21.15.159:0/4192042614 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a8c19f900 con 0x7f6a8c104010 2024-09-19T07:31:00.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.352+0000 7f6a89600640 1 -- 172.21.15.159:0/4192042614 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a7c004020 con 0x7f6a8c104010 2024-09-19T07:31:00.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.352+0000 7f6a89600640 1 -- 172.21.15.159:0/4192042614 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00ba70 con 0x7f6a8c104010 2024-09-19T07:31:00.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.352+0000 7f6a93200640 1 -- 172.21.15.159:0/4192042614 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a8c1a71e0 con 0x7f6a8c104010 2024-09-19T07:31:00.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.354+0000 7f6a93200640 1 -- 172.21.15.159:0/4192042614 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a8c1043f0 con 0x7f6a8c104010 2024-09-19T07:31:00.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.355+0000 7f6a89600640 1 -- 172.21.15.159:0/4192042614 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f6a7c0041d0 con 0x7f6a8c104010 2024-09-19T07:31:00.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.357+0000 7f6a89600640 1 --2- 172.21.15.159:0/4192042614 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6a700a3240 0x7f6a700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:00.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.357+0000 7f6a89600640 1 -- 172.21.15.159:0/4192042614 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7f6a7c0cfb90 con 0x7f6a8c104010 2024-09-19T07:31:00.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.357+0000 7f6a83400640 1 --2- 172.21.15.159:0/4192042614 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6a700a3240 0x7f6a700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:00.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.358+0000 7f6a83400640 1 --2- 172.21.15.159:0/4192042614 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6a700a3240 0x7f6a700a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6a740057c0 tx=0x7f6a740056d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:00.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.359+0000 7f6a89600640 1 -- 172.21.15.159:0/4192042614 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6a7c09cb40 con 0x7f6a8c104010 2024-09-19T07:31:00.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:00.537+0000 7f6a93200640 1 -- 172.21.15.159:0/4192042614 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f6a8c10c380 con 0x7f6a8c104010 2024-09-19T07:31:01.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:01.410+0000 7f6a89600640 1 -- 172.21.15.159:0/4192042614 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v86) ==== 171+0+0 (secure 0 0 0) 0x7f6a7c09c440 con 0x7f6a8c104010 2024-09-19T07:31:01.412 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' created 2024-09-19T07:31:01.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:01.415+0000 7f6a93200640 1 -- 172.21.15.159:0/4192042614 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6a700a3240 msgr2=0x7f6a700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:01.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:01.415+0000 7f6a93200640 1 --2- 172.21.15.159:0/4192042614 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6a700a3240 0x7f6a700a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6a740057c0 tx=0x7f6a740056d0 comp rx=0 tx=0).stop 2024-09-19T07:31:01.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:01.415+0000 7f6a93200640 1 -- 172.21.15.159:0/4192042614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a8c104010 msgr2=0x7f6a8c19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:01.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:01.415+0000 7f6a93200640 1 --2- 172.21.15.159:0/4192042614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a8c104010 0x7f6a8c19a660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009fd0 tx=0x7f6a7c00c880 comp rx=0 tx=0).stop 2024-09-19T07:31:01.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:01.416+0000 7f6a93200640 1 -- 172.21.15.159:0/4192042614 shutdown_connections 2024-09-19T07:31:01.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:01.416+0000 7f6a93200640 1 --2- 172.21.15.159:0/4192042614 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6a700a3240 0x7f6a700a5700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:01.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:01.416+0000 7f6a93200640 1 --2- 172.21.15.159:0/4192042614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a8c109140 0x7f6a8c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:01.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:01.416+0000 7f6a93200640 1 --2- 172.21.15.159:0/4192042614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a8c1049c0 0x7f6a8c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:01.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:01.416+0000 7f6a93200640 1 --2- 172.21.15.159:0/4192042614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a8c104010 0x7f6a8c19a660 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:01.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:01.416+0000 7f6a93200640 1 -- 172.21.15.159:0/4192042614 >> 172.21.15.159:0/4192042614 conn(0x7f6a8c0ffa50 msgr2=0x7f6a8c100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:01.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:01.416+0000 7f6a93200640 1 -- 172.21.15.159:0/4192042614 shutdown_connections 2024-09-19T07:31:01.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:01.417+0000 7f6a93200640 1 -- 172.21.15.159:0/4192042614 wait complete. 2024-09-19T07:31:01.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:01 smithi203 ceph-mon[35335]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T07:31:01.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:01 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/4192042614' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-19T07:31:01.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:01 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-19T07:31:01.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:01 smithi203 ceph-mon[35335]: pgmap v300: 33 pgs: 3 creating+peering, 20 unknown, 10 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:01.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:01 smithi189 ceph-mon[34851]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T07:31:01.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:01 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/4192042614' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-19T07:31:01.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:01 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-19T07:31:01.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:01 smithi189 ceph-mon[34851]: pgmap v300: 33 pgs: 3 creating+peering, 20 unknown, 10 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:01.852 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:01 smithi159 ceph-mon[30472]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T07:31:01.852 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:01 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/4192042614' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-19T07:31:01.853 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:01 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-19T07:31:01.853 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:01 smithi159 ceph-mon[30472]: pgmap v300: 33 pgs: 3 creating+peering, 20 unknown, 10 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:02.259 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T07:31:02.646 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:02.785 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:02 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-19T07:31:02.785 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:02 smithi159 ceph-mon[30472]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T07:31:02.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:02 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-19T07:31:02.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:02 smithi203 ceph-mon[35335]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T07:31:02.805 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:02 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-19T07:31:02.805 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:02 smithi189 ceph-mon[34851]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T07:31:03.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.282+0000 7f9518a00640 1 -- 172.21.15.159:0/1499359973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95141019c0 msgr2=0x7f9514101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:03.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.282+0000 7f9518a00640 1 --2- 172.21.15.159:0/1499359973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95141019c0 0x7f9514101e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9500009930 tx=0x7f950002a420 comp rx=0 tx=0).stop 2024-09-19T07:31:03.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.282+0000 7f9518a00640 1 -- 172.21.15.159:0/1499359973 shutdown_connections 2024-09-19T07:31:03.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.282+0000 7f9518a00640 1 --2- 172.21.15.159:0/1499359973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9514102360 0x7f951410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:03.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.282+0000 7f9518a00640 1 --2- 172.21.15.159:0/1499359973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95141019c0 0x7f9514101e20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:03.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.282+0000 7f9518a00640 1 --2- 172.21.15.159:0/1499359973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95141079c0 0x7f9514107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:03.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.282+0000 7f9518a00640 1 -- 172.21.15.159:0/1499359973 >> 172.21.15.159:0/1499359973 conn(0x7f95140fd740 msgr2=0x7f95140ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:03.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.282+0000 7f9518a00640 1 -- 172.21.15.159:0/1499359973 shutdown_connections 2024-09-19T07:31:03.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.282+0000 7f9518a00640 1 -- 172.21.15.159:0/1499359973 wait complete. 2024-09-19T07:31:03.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.283+0000 7f9518a00640 1 Processor -- start 2024-09-19T07:31:03.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.283+0000 7f9518a00640 1 -- start start 2024-09-19T07:31:03.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.284+0000 7f9518a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95141019c0 0x7f951419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:03.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.284+0000 7f9518a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9514102360 0x7f951419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:03.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.284+0000 7f9518a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95141079c0 0x7f951419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:03.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.284+0000 7f9518a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9514111e10 con 0x7f95141019c0 2024-09-19T07:31:03.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.284+0000 7f9518a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9514111c90 con 0x7f9514102360 2024-09-19T07:31:03.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.284+0000 7f9518a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9514111f90 con 0x7f95141079c0 2024-09-19T07:31:03.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.284+0000 7f9512a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9514102360 0x7f951419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:03.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.284+0000 7f9513400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95141019c0 0x7f951419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:03.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.284+0000 7f9512a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9514102360 0x7f951419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:54630/0 (socket says 172.21.15.159:54630) 2024-09-19T07:31:03.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.284+0000 7f9512a00640 1 -- 172.21.15.159:0/521523510 learned_addr learned my addr 172.21.15.159:0/521523510 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:03.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.284+0000 7f9513e00640 1 --2- 172.21.15.159:0/521523510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95141079c0 0x7f951419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:03.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.285+0000 7f9512a00640 1 -- 172.21.15.159:0/521523510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95141079c0 msgr2=0x7f951419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:03.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.285+0000 7f9512a00640 1 --2- 172.21.15.159:0/521523510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95141079c0 0x7f951419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:03.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.285+0000 7f9512a00640 1 -- 172.21.15.159:0/521523510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95141019c0 msgr2=0x7f951419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:03.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.285+0000 7f9512a00640 1 --2- 172.21.15.159:0/521523510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95141019c0 0x7f951419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:03.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.285+0000 7f9512a00640 1 -- 172.21.15.159:0/521523510 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951419f490 con 0x7f9514102360 2024-09-19T07:31:03.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.285+0000 7f9513e00640 1 --2- 172.21.15.159:0/521523510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95141079c0 0x7f951419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:31:03.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.285+0000 7f9513400640 1 --2- 172.21.15.159:0/521523510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95141019c0 0x7f951419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:31:03.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.285+0000 7f9512a00640 1 --2- 172.21.15.159:0/521523510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9514102360 0x7f951419aa20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f950002a940 tx=0x7f9500003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:03.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.286+0000 7f950be00640 1 -- 172.21.15.159:0/521523510 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950003f070 con 0x7f9514102360 2024-09-19T07:31:03.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.286+0000 7f9518a00640 1 -- 172.21.15.159:0/521523510 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951419f720 con 0x7f9514102360 2024-09-19T07:31:03.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.286+0000 7f950be00640 1 -- 172.21.15.159:0/521523510 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9500031c90 con 0x7f9514102360 2024-09-19T07:31:03.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.286+0000 7f950be00640 1 -- 172.21.15.159:0/521523510 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500035030 con 0x7f9514102360 2024-09-19T07:31:03.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.286+0000 7f9518a00640 1 -- 172.21.15.159:0/521523510 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95141a7060 con 0x7f9514102360 2024-09-19T07:31:03.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.287+0000 7f9518a00640 1 -- 172.21.15.159:0/521523510 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9514103100 con 0x7f9514102360 2024-09-19T07:31:03.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.289+0000 7f950be00640 1 -- 172.21.15.159:0/521523510 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9500042070 con 0x7f9514102360 2024-09-19T07:31:03.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.291+0000 7f950be00640 1 --2- 172.21.15.159:0/521523510 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f94f80a3240 0x7f94f80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:03.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.291+0000 7f950be00640 1 -- 172.21.15.159:0/521523510 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7879+0+0 (secure 0 0 0) 0x7f95000f05b0 con 0x7f9514102360 2024-09-19T07:31:03.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.291+0000 7f9513400640 1 --2- 172.21.15.159:0/521523510 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f94f80a3240 0x7f94f80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:03.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.292+0000 7f9513400640 1 --2- 172.21.15.159:0/521523510 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f94f80a3240 0x7f94f80a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9504009f40 tx=0x7f9504005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:03.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.292+0000 7f950be00640 1 -- 172.21.15.159:0/521523510 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f95000bcd90 con 0x7f9514102360 2024-09-19T07:31:03.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:03.516+0000 7f9518a00640 1 -- 172.21.15.159:0/521523510 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f95141a7230 con 0x7f9514102360 2024-09-19T07:31:03.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:03 smithi203 ceph-mon[35335]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T07:31:03.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:03 smithi203 ceph-mon[35335]: pgmap v303: 97 pgs: 9 creating+peering, 56 unknown, 32 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:03.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:03 smithi189 ceph-mon[34851]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T07:31:03.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:03 smithi189 ceph-mon[34851]: pgmap v303: 97 pgs: 9 creating+peering, 56 unknown, 32 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:03.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:03 smithi159 ceph-mon[30472]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T07:31:03.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:03 smithi159 ceph-mon[30472]: pgmap v303: 97 pgs: 9 creating+peering, 56 unknown, 32 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:04.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:04.431+0000 7f950be00640 1 -- 172.21.15.159:0/521523510 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f95000c1090 con 0x7f9514102360 2024-09-19T07:31:04.432 INFO:teuthology.orchestra.run.smithi159.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T07:31:04.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:04.435+0000 7f9518a00640 1 -- 172.21.15.159:0/521523510 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f94f80a3240 msgr2=0x7f94f80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:04.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:04.435+0000 7f9518a00640 1 --2- 172.21.15.159:0/521523510 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f94f80a3240 0x7f94f80a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9504009f40 tx=0x7f9504005f90 comp rx=0 tx=0).stop 2024-09-19T07:31:04.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:04.435+0000 7f9518a00640 1 -- 172.21.15.159:0/521523510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9514102360 msgr2=0x7f951419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:04.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:04.435+0000 7f9518a00640 1 --2- 172.21.15.159:0/521523510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9514102360 0x7f951419aa20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f950002a940 tx=0x7f9500003b20 comp rx=0 tx=0).stop 2024-09-19T07:31:04.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:04.435+0000 7f9518a00640 1 -- 172.21.15.159:0/521523510 shutdown_connections 2024-09-19T07:31:04.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:04.435+0000 7f9518a00640 1 --2- 172.21.15.159:0/521523510 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f94f80a3240 0x7f94f80a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:04.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:04.435+0000 7f9518a00640 1 --2- 172.21.15.159:0/521523510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95141079c0 0x7f951419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:04.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:04.435+0000 7f9518a00640 1 --2- 172.21.15.159:0/521523510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9514102360 0x7f951419aa20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:04.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:04.435+0000 7f9518a00640 1 --2- 172.21.15.159:0/521523510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95141019c0 0x7f951419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:04.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:04.435+0000 7f9518a00640 1 -- 172.21.15.159:0/521523510 >> 172.21.15.159:0/521523510 conn(0x7f95140fd740 msgr2=0x7f951410cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:04.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:04.436+0000 7f9518a00640 1 -- 172.21.15.159:0/521523510 shutdown_connections 2024-09-19T07:31:04.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:04.437+0000 7f9518a00640 1 -- 172.21.15.159:0/521523510 wait complete. 2024-09-19T07:31:04.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:04 smithi203 ceph-mon[35335]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T07:31:04.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:04 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/521523510' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T07:31:04.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:04 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T07:31:04.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:04 smithi203 ceph-mon[35335]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T07:31:04.805 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:04 smithi189 ceph-mon[34851]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T07:31:04.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:04 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/521523510' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T07:31:04.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:04 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T07:31:04.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:04 smithi189 ceph-mon[34851]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T07:31:04.878 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:04 smithi159 ceph-mon[30472]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T07:31:04.879 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:04 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/521523510' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T07:31:04.879 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:04 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T07:31:04.879 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:04 smithi159 ceph-mon[30472]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T07:31:05.427 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-19T07:31:05.427 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-19T07:31:05.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:05 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T07:31:05.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:05 smithi203 ceph-mon[35335]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T07:31:05.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:05 smithi203 ceph-mon[35335]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T07:31:05.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:05 smithi203 ceph-mon[35335]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T07:31:05.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:05 smithi203 ceph-mon[35335]: pgmap v306: 97 pgs: 6 creating+peering, 13 unknown, 78 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-09-19T07:31:05.798 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:05.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:05 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T07:31:05.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:05 smithi189 ceph-mon[34851]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T07:31:05.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:05 smithi189 ceph-mon[34851]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T07:31:05.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:05 smithi189 ceph-mon[34851]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T07:31:05.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:05 smithi189 ceph-mon[34851]: pgmap v306: 97 pgs: 6 creating+peering, 13 unknown, 78 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-09-19T07:31:05.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:05 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T07:31:05.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:05 smithi159 ceph-mon[30472]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T07:31:05.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:05 smithi159 ceph-mon[30472]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T07:31:05.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:05 smithi159 ceph-mon[30472]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T07:31:05.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:05 smithi159 ceph-mon[30472]: pgmap v306: 97 pgs: 6 creating+peering, 13 unknown, 78 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-09-19T07:31:06.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.449+0000 7fba3ca00640 1 -- 172.21.15.159:0/611592628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba380745c0 msgr2=0x7fba380749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:06.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.449+0000 7fba3ca00640 1 --2- 172.21.15.159:0/611592628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba380745c0 0x7fba380749a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fba28009900 tx=0x7fba2802a480 comp rx=0 tx=0).stop 2024-09-19T07:31:06.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.450+0000 7fba3ca00640 1 -- 172.21.15.159:0/611592628 shutdown_connections 2024-09-19T07:31:06.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.450+0000 7fba3ca00640 1 --2- 172.21.15.159:0/611592628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba3810e420 0x7fba38110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.450+0000 7fba3ca00640 1 --2- 172.21.15.159:0/611592628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba38074ee0 0x7fba3810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.450+0000 7fba3ca00640 1 --2- 172.21.15.159:0/611592628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba380745c0 0x7fba380749a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.450+0000 7fba3ca00640 1 -- 172.21.15.159:0/611592628 >> 172.21.15.159:0/611592628 conn(0x7fba380fd8c0 msgr2=0x7fba380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.450+0000 7fba3ca00640 1 -- 172.21.15.159:0/611592628 shutdown_connections 2024-09-19T07:31:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.450+0000 7fba3ca00640 1 -- 172.21.15.159:0/611592628 wait complete. 2024-09-19T07:31:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.451+0000 7fba3ca00640 1 Processor -- start 2024-09-19T07:31:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.451+0000 7fba3ca00640 1 -- start start 2024-09-19T07:31:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.451+0000 7fba3ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba380745c0 0x7fba3819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.451+0000 7fba3ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba38074ee0 0x7fba3819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.451+0000 7fba3ca00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba3810e420 0x7fba3819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.451+0000 7fba3ca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fba381141c0 con 0x7fba38074ee0 2024-09-19T07:31:06.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.451+0000 7fba3ca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fba38114040 con 0x7fba380745c0 2024-09-19T07:31:06.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.451+0000 7fba3ca00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fba38114340 con 0x7fba3810e420 2024-09-19T07:31:06.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.451+0000 7fba35e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba380745c0 0x7fba3819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:06.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.451+0000 7fba35400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba38074ee0 0x7fba3819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:06.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.451+0000 7fba35e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba380745c0 0x7fba3819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:54656/0 (socket says 172.21.15.159:54656) 2024-09-19T07:31:06.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.451+0000 7fba35e00640 1 -- 172.21.15.159:0/329794637 learned_addr learned my addr 172.21.15.159:0/329794637 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:06.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.451+0000 7fba36800640 1 --2- 172.21.15.159:0/329794637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba3810e420 0x7fba3819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:06.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.452+0000 7fba35400640 1 -- 172.21.15.159:0/329794637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba3810e420 msgr2=0x7fba3819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:06.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.452+0000 7fba35400640 1 --2- 172.21.15.159:0/329794637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba3810e420 0x7fba3819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:06.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.452+0000 7fba35400640 1 -- 172.21.15.159:0/329794637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba380745c0 msgr2=0x7fba3819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:06.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.452+0000 7fba35400640 1 --2- 172.21.15.159:0/329794637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba380745c0 0x7fba3819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:06.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.452+0000 7fba35400640 1 -- 172.21.15.159:0/329794637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3819f620 con 0x7fba38074ee0 2024-09-19T07:31:06.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.452+0000 7fba35e00640 1 --2- 172.21.15.159:0/329794637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba380745c0 0x7fba3819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:31:06.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.452+0000 7fba35400640 1 --2- 172.21.15.159:0/329794637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba38074ee0 0x7fba3819abb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fba2400e9f0 tx=0x7fba2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:06.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.452+0000 7fba2ea00640 1 -- 172.21.15.159:0/329794637 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2400ccc0 con 0x7fba38074ee0 2024-09-19T07:31:06.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.452+0000 7fba2ea00640 1 -- 172.21.15.159:0/329794637 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba24010910 con 0x7fba38074ee0 2024-09-19T07:31:06.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.452+0000 7fba2ea00640 1 -- 172.21.15.159:0/329794637 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24018960 con 0x7fba38074ee0 2024-09-19T07:31:06.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.452+0000 7fba3ca00640 1 -- 172.21.15.159:0/329794637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3819f910 con 0x7fba38074ee0 2024-09-19T07:31:06.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.453+0000 7fba3ca00640 1 -- 172.21.15.159:0/329794637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba381a71f0 con 0x7fba38074ee0 2024-09-19T07:31:06.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.454+0000 7fba3ca00640 1 -- 172.21.15.159:0/329794637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba380749a0 con 0x7fba38074ee0 2024-09-19T07:31:06.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.456+0000 7fba2ea00640 1 -- 172.21.15.159:0/329794637 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fba24010ac0 con 0x7fba38074ee0 2024-09-19T07:31:06.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.458+0000 7fba2ea00640 1 --2- 172.21.15.159:0/329794637 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fba1c0a32c0 0x7fba1c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:06.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.458+0000 7fba2ea00640 1 -- 172.21.15.159:0/329794637 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fba240cffe0 con 0x7fba38074ee0 2024-09-19T07:31:06.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.458+0000 7fba35e00640 1 --2- 172.21.15.159:0/329794637 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fba1c0a32c0 0x7fba1c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:06.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.459+0000 7fba35e00640 1 --2- 172.21.15.159:0/329794637 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fba1c0a32c0 0x7fba1c0a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fba280096c0 tx=0x7fba280046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:06.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.460+0000 7fba2ea00640 1 -- 172.21.15.159:0/329794637 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fba2409c780 con 0x7fba38074ee0 2024-09-19T07:31:06.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:06 smithi159 ceph-mon[30472]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T07:31:06.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:06 smithi159 ceph-mon[30472]: Cluster is now healthy 2024-09-19T07:31:06.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:06 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] up:active 2024-09-19T07:31:06.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:06 smithi159 ceph-mon[30472]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:31:06.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:06 smithi159 ceph-mon[30472]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:31:06.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:06.646+0000 7fba3ca00640 1 -- 172.21.15.159:0/329794637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fba3802d7c0 con 0x7fba38074ee0 2024-09-19T07:31:06.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:06 smithi203 ceph-mon[35335]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T07:31:06.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:06 smithi203 ceph-mon[35335]: Cluster is now healthy 2024-09-19T07:31:06.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:06 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] up:active 2024-09-19T07:31:06.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:06 smithi203 ceph-mon[35335]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:31:06.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:06 smithi203 ceph-mon[35335]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:31:06.805 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:06 smithi189 ceph-mon[34851]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T07:31:06.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:06 smithi189 ceph-mon[34851]: Cluster is now healthy 2024-09-19T07:31:06.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:06 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] up:active 2024-09-19T07:31:06.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:06 smithi189 ceph-mon[34851]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:31:06.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:06 smithi189 ceph-mon[34851]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:31:07.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:07.449+0000 7fba2ea00640 1 -- 172.21.15.159:0/329794637 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v90) ==== 157+0+0 (secure 0 0 0) 0x7fba240a0a80 con 0x7fba38074ee0 2024-09-19T07:31:07.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:07.452+0000 7fba3ca00640 1 -- 172.21.15.159:0/329794637 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fba1c0a32c0 msgr2=0x7fba1c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:07.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:07.452+0000 7fba3ca00640 1 --2- 172.21.15.159:0/329794637 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fba1c0a32c0 0x7fba1c0a5780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fba280096c0 tx=0x7fba280046c0 comp rx=0 tx=0).stop 2024-09-19T07:31:07.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:07.452+0000 7fba3ca00640 1 -- 172.21.15.159:0/329794637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba38074ee0 msgr2=0x7fba3819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:07.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:07.452+0000 7fba3ca00640 1 --2- 172.21.15.159:0/329794637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba38074ee0 0x7fba3819abb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fba2400e9f0 tx=0x7fba2400eeb0 comp rx=0 tx=0).stop 2024-09-19T07:31:07.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:07.452+0000 7fba3ca00640 1 -- 172.21.15.159:0/329794637 shutdown_connections 2024-09-19T07:31:07.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:07.452+0000 7fba3ca00640 1 --2- 172.21.15.159:0/329794637 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fba1c0a32c0 0x7fba1c0a5780 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:07.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:07.452+0000 7fba3ca00640 1 --2- 172.21.15.159:0/329794637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba3810e420 0x7fba3819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:07.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:07.452+0000 7fba3ca00640 1 --2- 172.21.15.159:0/329794637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba38074ee0 0x7fba3819abb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:07.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:07.452+0000 7fba3ca00640 1 --2- 172.21.15.159:0/329794637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba380745c0 0x7fba3819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:07.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:07.452+0000 7fba3ca00640 1 -- 172.21.15.159:0/329794637 >> 172.21.15.159:0/329794637 conn(0x7fba380fd8c0 msgr2=0x7fba380fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:07.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:07.453+0000 7fba3ca00640 1 -- 172.21.15.159:0/329794637 shutdown_connections 2024-09-19T07:31:07.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:07.453+0000 7fba3ca00640 1 -- 172.21.15.159:0/329794637 wait complete. 2024-09-19T07:31:07.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:07 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/329794637' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-19T07:31:07.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:07 smithi203 ceph-mon[35335]: pgmap v307: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 7 op/s 2024-09-19T07:31:07.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:07 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/329794637' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-19T07:31:07.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:07 smithi189 ceph-mon[34851]: pgmap v307: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 7 op/s 2024-09-19T07:31:07.862 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:07 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/329794637' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-19T07:31:07.862 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:07 smithi159 ceph-mon[30472]: pgmap v307: 97 pgs: 6 creating+peering, 91 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 7 op/s 2024-09-19T07:31:08.397 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T07:31:08.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:08 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/329794637' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-19T07:31:08.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:08 smithi159 ceph-mon[30472]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T07:31:08.770 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:08.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:08 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/329794637' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-19T07:31:08.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:08 smithi203 ceph-mon[35335]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T07:31:08.805 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:08 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/329794637' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-19T07:31:08.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:08 smithi189 ceph-mon[34851]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T07:31:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.376+0000 7fa369c00640 1 -- 172.21.15.159:0/3434204433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa364102790 msgr2=0x7fa3641067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.376+0000 7fa369c00640 1 --2- 172.21.15.159:0/3434204433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa364102790 0x7fa3641067e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009900 tx=0x7fa34c02a410 comp rx=0 tx=0).stop 2024-09-19T07:31:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.376+0000 7fa369c00640 1 -- 172.21.15.159:0/3434204433 shutdown_connections 2024-09-19T07:31:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.376+0000 7fa369c00640 1 --2- 172.21.15.159:0/3434204433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa364106f10 0x7fa36410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:09.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.376+0000 7fa369c00640 1 --2- 172.21.15.159:0/3434204433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa364102790 0x7fa3641067e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:09.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.376+0000 7fa369c00640 1 --2- 172.21.15.159:0/3434204433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa364101de0 0x7fa3641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:09.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.376+0000 7fa369c00640 1 -- 172.21.15.159:0/3434204433 >> 172.21.15.159:0/3434204433 conn(0x7fa3640fd8c0 msgr2=0x7fa3640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:09.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.376+0000 7fa369c00640 1 -- 172.21.15.159:0/3434204433 shutdown_connections 2024-09-19T07:31:09.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.376+0000 7fa369c00640 1 -- 172.21.15.159:0/3434204433 wait complete. 2024-09-19T07:31:09.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.377+0000 7fa369c00640 1 Processor -- start 2024-09-19T07:31:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.377+0000 7fa369c00640 1 -- start start 2024-09-19T07:31:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.377+0000 7fa369c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa364101de0 0x7fa36419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.377+0000 7fa369c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa364102790 0x7fa36419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.377+0000 7fa369c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa364106f10 0x7fa36419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.377+0000 7fa369c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa364111f30 con 0x7fa364102790 2024-09-19T07:31:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.377+0000 7fa369c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa364111db0 con 0x7fa364101de0 2024-09-19T07:31:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.377+0000 7fa369c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641120b0 con 0x7fa364106f10 2024-09-19T07:31:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.378+0000 7fa35a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa364102790 0x7fa36419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.378+0000 7fa35a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa364102790 0x7fa36419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40128/0 (socket says 172.21.15.159:40128) 2024-09-19T07:31:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.378+0000 7fa35a000640 1 -- 172.21.15.159:0/2224221455 learned_addr learned my addr 172.21.15.159:0/2224221455 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.378+0000 7fa363400640 1 --2- 172.21.15.159:0/2224221455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa364106f10 0x7fa36419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.378+0000 7fa362a00640 1 --2- 172.21.15.159:0/2224221455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa364101de0 0x7fa36419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:09.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.378+0000 7fa363400640 1 -- 172.21.15.159:0/2224221455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa364101de0 msgr2=0x7fa36419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:09.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.378+0000 7fa363400640 1 --2- 172.21.15.159:0/2224221455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa364101de0 0x7fa36419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:09.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.378+0000 7fa363400640 1 -- 172.21.15.159:0/2224221455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa364102790 msgr2=0x7fa36419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:09.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.378+0000 7fa363400640 1 --2- 172.21.15.159:0/2224221455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa364102790 0x7fa36419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:09.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.378+0000 7fa363400640 1 -- 172.21.15.159:0/2224221455 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36419f5e0 con 0x7fa364106f10 2024-09-19T07:31:09.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.378+0000 7fa35a000640 1 --2- 172.21.15.159:0/2224221455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa364102790 0x7fa36419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:31:09.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.378+0000 7fa363400640 1 --2- 172.21.15.159:0/2224221455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa364106f10 0x7fa36419ef00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009fd0 tx=0x7fa35c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:09.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.379+0000 7fa35be00640 1 -- 172.21.15.159:0/2224221455 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c00ce20 con 0x7fa364106f10 2024-09-19T07:31:09.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.379+0000 7fa369c00640 1 -- 172.21.15.159:0/2224221455 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36419f8d0 con 0x7fa364106f10 2024-09-19T07:31:09.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.379+0000 7fa35be00640 1 -- 172.21.15.159:0/2224221455 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa35c005020 con 0x7fa364106f10 2024-09-19T07:31:09.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.379+0000 7fa35be00640 1 -- 172.21.15.159:0/2224221455 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c010720 con 0x7fa364106f10 2024-09-19T07:31:09.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.379+0000 7fa369c00640 1 -- 172.21.15.159:0/2224221455 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3641a71b0 con 0x7fa364106f10 2024-09-19T07:31:09.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.380+0000 7fa369c00640 1 -- 172.21.15.159:0/2224221455 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3641021c0 con 0x7fa364106f10 2024-09-19T07:31:09.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.382+0000 7fa35be00640 1 -- 172.21.15.159:0/2224221455 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa35c005540 con 0x7fa364106f10 2024-09-19T07:31:09.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.383+0000 7fa35be00640 1 --2- 172.21.15.159:0/2224221455 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa3480a32c0 0x7fa3480a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:09.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.383+0000 7fa35be00640 1 -- 172.21.15.159:0/2224221455 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fa35c0cff10 con 0x7fa364106f10 2024-09-19T07:31:09.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.383+0000 7fa362a00640 1 --2- 172.21.15.159:0/2224221455 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa3480a32c0 0x7fa3480a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:09.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.384+0000 7fa362a00640 1 --2- 172.21.15.159:0/2224221455 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa3480a32c0 0x7fa3480a5780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa3540095f0 tx=0x7fa354005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:09.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.385+0000 7fa35be00640 1 -- 172.21.15.159:0/2224221455 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa35c09cba0 con 0x7fa364106f10 2024-09-19T07:31:09.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:09.565+0000 7fa369c00640 1 -- 172.21.15.159:0/2224221455 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa3641a7490 con 0x7fa364106f10 2024-09-19T07:31:09.567 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:09 smithi159 ceph-mon[30472]: pgmap v309: 97 pgs: 97 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-19T07:31:09.742 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:09 smithi189 ceph-mon[34851]: pgmap v309: 97 pgs: 97 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-19T07:31:09.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:09 smithi203 ceph-mon[35335]: pgmap v309: 97 pgs: 97 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-19T07:31:10.056 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:31:09 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-b[64691]: 2024-09-19T07:31:09.741+0000 7f8c4eb2e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T07:31:10.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:10 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/2224221455' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T07:31:10.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:10 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T07:31:10.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:10 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/2224221455' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T07:31:10.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:10 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T07:31:10.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:10 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2224221455' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T07:31:10.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:10 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T07:31:11.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:11.482+0000 7fa35be00640 1 -- 172.21.15.159:0/2224221455 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v92) ==== 228+0+0 (secure 0 0 0) 0x7fa35c09c4a0 con 0x7fa364106f10 2024-09-19T07:31:11.484 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data_ec' created 2024-09-19T07:31:11.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:11.487+0000 7fa369c00640 1 -- 172.21.15.159:0/2224221455 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa3480a32c0 msgr2=0x7fa3480a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:11.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:11.487+0000 7fa369c00640 1 --2- 172.21.15.159:0/2224221455 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa3480a32c0 0x7fa3480a5780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa3540095f0 tx=0x7fa354005f90 comp rx=0 tx=0).stop 2024-09-19T07:31:11.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:11.487+0000 7fa369c00640 1 -- 172.21.15.159:0/2224221455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa364106f10 msgr2=0x7fa36419ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:11.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:11.487+0000 7fa369c00640 1 --2- 172.21.15.159:0/2224221455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa364106f10 0x7fa36419ef00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009fd0 tx=0x7fa35c00ef10 comp rx=0 tx=0).stop 2024-09-19T07:31:11.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:11.487+0000 7fa369c00640 1 -- 172.21.15.159:0/2224221455 shutdown_connections 2024-09-19T07:31:11.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:11.487+0000 7fa369c00640 1 --2- 172.21.15.159:0/2224221455 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa3480a32c0 0x7fa3480a5780 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:11.487+0000 7fa369c00640 1 --2- 172.21.15.159:0/2224221455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa364106f10 0x7fa36419ef00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:11.487+0000 7fa369c00640 1 --2- 172.21.15.159:0/2224221455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa364102790 0x7fa36419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:11.487+0000 7fa369c00640 1 --2- 172.21.15.159:0/2224221455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa364101de0 0x7fa36419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:11.487+0000 7fa369c00640 1 -- 172.21.15.159:0/2224221455 >> 172.21.15.159:0/2224221455 conn(0x7fa3640fd8c0 msgr2=0x7fa3640febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:11.488+0000 7fa369c00640 1 -- 172.21.15.159:0/2224221455 shutdown_connections 2024-09-19T07:31:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:11.488+0000 7fa369c00640 1 -- 172.21.15.159:0/2224221455 wait complete. 2024-09-19T07:31:11.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:11 smithi203 ceph-mon[35335]: pgmap v311: 97 pgs: 97 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-19T07:31:11.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:11 smithi203 ceph-mon[35335]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T07:31:11.805 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:11 smithi189 ceph-mon[34851]: pgmap v311: 97 pgs: 97 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-19T07:31:11.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:11 smithi189 ceph-mon[34851]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T07:31:11.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:11 smithi159 ceph-mon[30472]: pgmap v311: 97 pgs: 97 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-19T07:31:11.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:11 smithi159 ceph-mon[30472]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T07:31:12.469 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-19T07:31:12.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:12 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T07:31:12.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:12 smithi203 ceph-mon[35335]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T07:31:12.805 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:12 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T07:31:12.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:12 smithi189 ceph-mon[34851]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T07:31:12.856 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:12.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:12 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T07:31:12.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:12 smithi159 ceph-mon[30472]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T07:31:13.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.439+0000 7fe3e1800640 1 -- 172.21.15.159:0/3756297384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3dc074590 msgr2=0x7fe3dc074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:13.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.439+0000 7fe3e1800640 1 --2- 172.21.15.159:0/3756297384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3dc074590 0x7fe3dc074970 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009ca0 tx=0x7fe3cc02a950 comp rx=0 tx=0).stop 2024-09-19T07:31:13.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.439+0000 7fe3e1800640 1 -- 172.21.15.159:0/3756297384 shutdown_connections 2024-09-19T07:31:13.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.439+0000 7fe3e1800640 1 --2- 172.21.15.159:0/3756297384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3dc112b90 0x7fe3dc114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:13.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.439+0000 7fe3e1800640 1 --2- 172.21.15.159:0/3756297384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3dc074eb0 0x7fe3dc112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:13.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.439+0000 7fe3e1800640 1 --2- 172.21.15.159:0/3756297384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3dc074590 0x7fe3dc074970 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:13.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.439+0000 7fe3e1800640 1 -- 172.21.15.159:0/3756297384 >> 172.21.15.159:0/3756297384 conn(0x7fe3dc0fd8c0 msgr2=0x7fe3dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:13.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.439+0000 7fe3e1800640 1 -- 172.21.15.159:0/3756297384 shutdown_connections 2024-09-19T07:31:13.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.440+0000 7fe3e1800640 1 -- 172.21.15.159:0/3756297384 wait complete. 2024-09-19T07:31:13.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.440+0000 7fe3e1800640 1 Processor -- start 2024-09-19T07:31:13.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.441+0000 7fe3e1800640 1 -- start start 2024-09-19T07:31:13.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.441+0000 7fe3e1800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3dc074590 0x7fe3dc1a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:13.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.441+0000 7fe3e1800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3dc074eb0 0x7fe3dc1a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:13.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.441+0000 7fe3e1800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3dc112b90 0x7fe3dc19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:13.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.441+0000 7fe3e1800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc118550 con 0x7fe3dc112b90 2024-09-19T07:31:13.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.441+0000 7fe3e1800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1183d0 con 0x7fe3dc074eb0 2024-09-19T07:31:13.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.441+0000 7fe3e1800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1186d0 con 0x7fe3dc074590 2024-09-19T07:31:13.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.441+0000 7fe3db400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3dc112b90 0x7fe3dc19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:13.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.441+0000 7fe3da000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3dc074eb0 0x7fe3dc1a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:13.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.441+0000 7fe3db400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3dc112b90 0x7fe3dc19e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48848/0 (socket says 172.21.15.159:48848) 2024-09-19T07:31:13.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.441+0000 7fe3db400640 1 -- 172.21.15.159:0/2260952127 learned_addr learned my addr 172.21.15.159:0/2260952127 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:13.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.441+0000 7fe3daa00640 1 --2- 172.21.15.159:0/2260952127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3dc074590 0x7fe3dc1a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:13.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.441+0000 7fe3db400640 1 -- 172.21.15.159:0/2260952127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3dc074590 msgr2=0x7fe3dc1a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:13.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.441+0000 7fe3db400640 1 --2- 172.21.15.159:0/2260952127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3dc074590 0x7fe3dc1a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:13.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.442+0000 7fe3db400640 1 -- 172.21.15.159:0/2260952127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3dc074eb0 msgr2=0x7fe3dc1a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:13.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.442+0000 7fe3db400640 1 --2- 172.21.15.159:0/2260952127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3dc074eb0 0x7fe3dc1a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:13.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.442+0000 7fe3db400640 1 -- 172.21.15.159:0/2260952127 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3dc19eff0 con 0x7fe3dc112b90 2024-09-19T07:31:13.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.442+0000 7fe3db400640 1 --2- 172.21.15.159:0/2260952127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3dc112b90 0x7fe3dc19e8c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400e970 tx=0x7fe3d400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:13.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.442+0000 7fe3d3400640 1 -- 172.21.15.159:0/2260952127 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d400ccc0 con 0x7fe3dc112b90 2024-09-19T07:31:13.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.442+0000 7fe3e1800640 1 -- 172.21.15.159:0/2260952127 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3dc19f2e0 con 0x7fe3dc112b90 2024-09-19T07:31:13.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.442+0000 7fe3d3400640 1 -- 172.21.15.159:0/2260952127 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d4005500 con 0x7fe3dc112b90 2024-09-19T07:31:13.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.442+0000 7fe3d3400640 1 -- 172.21.15.159:0/2260952127 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4010690 con 0x7fe3dc112b90 2024-09-19T07:31:13.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.442+0000 7fe3e1800640 1 -- 172.21.15.159:0/2260952127 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3dc1ab5d0 con 0x7fe3dc112b90 2024-09-19T07:31:13.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.443+0000 7fe3e1800640 1 -- 172.21.15.159:0/2260952127 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3dc074970 con 0x7fe3dc112b90 2024-09-19T07:31:13.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.446+0000 7fe3d3400640 1 -- 172.21.15.159:0/2260952127 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fe3d40046c0 con 0x7fe3dc112b90 2024-09-19T07:31:13.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.447+0000 7fe3d3400640 1 --2- 172.21.15.159:0/2260952127 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe3c00a3240 0x7fe3c00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:13.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.447+0000 7fe3d3400640 1 -- 172.21.15.159:0/2260952127 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8770+0+0 (secure 0 0 0) 0x7fe3d40d00b0 con 0x7fe3dc112b90 2024-09-19T07:31:13.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.447+0000 7fe3daa00640 1 --2- 172.21.15.159:0/2260952127 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe3c00a3240 0x7fe3c00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:13.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.448+0000 7fe3daa00640 1 --2- 172.21.15.159:0/2260952127 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe3c00a3240 0x7fe3c00a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc000f80 tx=0x7fe3cc004a60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:13.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.449+0000 7fe3d3400640 1 -- 172.21.15.159:0/2260952127 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe3d4054210 con 0x7fe3dc112b90 2024-09-19T07:31:13.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:13.631+0000 7fe3e1800640 1 -- 172.21.15.159:0/2260952127 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fe3dc1ab920 con 0x7fe3dc112b90 2024-09-19T07:31:13.633 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:13 smithi159 ceph-mon[30472]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T07:31:13.633 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:13 smithi159 ceph-mon[30472]: pgmap v314: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 0 op/s 2024-09-19T07:31:13.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:13 smithi203 ceph-mon[35335]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T07:31:13.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:13 smithi203 ceph-mon[35335]: pgmap v314: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 0 op/s 2024-09-19T07:31:13.805 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:13 smithi189 ceph-mon[34851]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T07:31:13.805 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:13 smithi189 ceph-mon[34851]: pgmap v314: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 0 op/s 2024-09-19T07:31:14.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:14.499+0000 7fe3d3400640 1 -- 172.21.15.159:0/2260952127 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v95) ==== 169+0+0 (secure 0 0 0) 0x7fe3d409cb50 con 0x7fe3dc112b90 2024-09-19T07:31:14.500 INFO:teuthology.orchestra.run.smithi159.stderr:set pool 4 allow_ec_overwrites to true 2024-09-19T07:31:14.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:14.502+0000 7fe3e1800640 1 -- 172.21.15.159:0/2260952127 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe3c00a3240 msgr2=0x7fe3c00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:14.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:14.502+0000 7fe3e1800640 1 --2- 172.21.15.159:0/2260952127 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe3c00a3240 0x7fe3c00a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc000f80 tx=0x7fe3cc004a60 comp rx=0 tx=0).stop 2024-09-19T07:31:14.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:14.502+0000 7fe3e1800640 1 -- 172.21.15.159:0/2260952127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3dc112b90 msgr2=0x7fe3dc19e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:14.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:14.502+0000 7fe3e1800640 1 --2- 172.21.15.159:0/2260952127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3dc112b90 0x7fe3dc19e8c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400e970 tx=0x7fe3d400ee30 comp rx=0 tx=0).stop 2024-09-19T07:31:14.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:14.502+0000 7fe3e1800640 1 -- 172.21.15.159:0/2260952127 shutdown_connections 2024-09-19T07:31:14.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:14.502+0000 7fe3e1800640 1 --2- 172.21.15.159:0/2260952127 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe3c00a3240 0x7fe3c00a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:14.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:14.502+0000 7fe3e1800640 1 --2- 172.21.15.159:0/2260952127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3dc112b90 0x7fe3dc19e8c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:14.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:14.502+0000 7fe3e1800640 1 --2- 172.21.15.159:0/2260952127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3dc074eb0 0x7fe3dc1a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:14.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:14.502+0000 7fe3e1800640 1 --2- 172.21.15.159:0/2260952127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3dc074590 0x7fe3dc1a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:14.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:14.502+0000 7fe3e1800640 1 -- 172.21.15.159:0/2260952127 >> 172.21.15.159:0/2260952127 conn(0x7fe3dc0fd8c0 msgr2=0x7fe3dc0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:14.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:14.502+0000 7fe3e1800640 1 -- 172.21.15.159:0/2260952127 shutdown_connections 2024-09-19T07:31:14.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:14.502+0000 7fe3e1800640 1 -- 172.21.15.159:0/2260952127 wait complete. 2024-09-19T07:31:14.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:14 smithi203 ceph-mon[35335]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T07:31:14.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:14 smithi203 ceph-mon[35335]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T07:31:14.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:14 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/2260952127' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-19T07:31:14.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:14 smithi189 ceph-mon[34851]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T07:31:14.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:14 smithi189 ceph-mon[34851]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T07:31:14.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:14 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/2260952127' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-19T07:31:14.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:14 smithi159 ceph-mon[30472]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T07:31:14.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:14 smithi159 ceph-mon[30472]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T07:31:14.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:14 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2260952127' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-19T07:31:15.492 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-19T07:31:15.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:15 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/2260952127' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-19T07:31:15.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:15 smithi203 ceph-mon[35335]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T07:31:15.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:15 smithi203 ceph-mon[35335]: pgmap v317: 129 pgs: 5 creating+peering, 15 unknown, 109 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:15.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:15 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/2260952127' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-19T07:31:15.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:15 smithi189 ceph-mon[34851]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T07:31:15.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:15 smithi189 ceph-mon[34851]: pgmap v317: 129 pgs: 5 creating+peering, 15 unknown, 109 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:15.864 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:15.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:15 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2260952127' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-19T07:31:15.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:15 smithi159 ceph-mon[30472]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T07:31:15.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:15 smithi159 ceph-mon[30472]: pgmap v317: 129 pgs: 5 creating+peering, 15 unknown, 109 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:16.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.437+0000 7fcca9a00640 1 -- 172.21.15.159:0/3549135773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcca41079c0 msgr2=0x7fcca4107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:16.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.437+0000 7fcca9a00640 1 --2- 172.21.15.159:0/3549135773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcca41079c0 0x7fcca4107da0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009930 tx=0x7fcc9402a420 comp rx=0 tx=0).stop 2024-09-19T07:31:16.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.437+0000 7fcca9a00640 1 -- 172.21.15.159:0/3549135773 shutdown_connections 2024-09-19T07:31:16.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.437+0000 7fcca9a00640 1 --2- 172.21.15.159:0/3549135773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcca4102360 0x7fcca410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:16.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.437+0000 7fcca9a00640 1 --2- 172.21.15.159:0/3549135773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca41019c0 0x7fcca4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:16.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.437+0000 7fcca9a00640 1 --2- 172.21.15.159:0/3549135773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcca41079c0 0x7fcca4107da0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:16.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.437+0000 7fcca9a00640 1 -- 172.21.15.159:0/3549135773 >> 172.21.15.159:0/3549135773 conn(0x7fcca40fd6e0 msgr2=0x7fcca40ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:16.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.437+0000 7fcca9a00640 1 -- 172.21.15.159:0/3549135773 shutdown_connections 2024-09-19T07:31:16.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.437+0000 7fcca9a00640 1 -- 172.21.15.159:0/3549135773 wait complete. 2024-09-19T07:31:16.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.438+0000 7fcca9a00640 1 Processor -- start 2024-09-19T07:31:16.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.438+0000 7fcca9a00640 1 -- start start 2024-09-19T07:31:16.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.439+0000 7fcca9a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcca41019c0 0x7fcca40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:16.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.439+0000 7fcca9a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcca4102360 0x7fcca4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:16.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.439+0000 7fcca9a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca41079c0 0x7fcca4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:16.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.439+0000 7fcca9a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcca4111e10 con 0x7fcca41019c0 2024-09-19T07:31:16.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.439+0000 7fcca9a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcca4111c90 con 0x7fcca4102360 2024-09-19T07:31:16.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.439+0000 7fcca9a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcca4111f90 con 0x7fcca41079c0 2024-09-19T07:31:16.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.439+0000 7fcca3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcca41019c0 0x7fcca40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:16.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.439+0000 7fcca3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcca41019c0 0x7fcca40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48866/0 (socket says 172.21.15.159:48866) 2024-09-19T07:31:16.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.439+0000 7fcca3e00640 1 -- 172.21.15.159:0/4208906398 learned_addr learned my addr 172.21.15.159:0/4208906398 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:16.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.439+0000 7fcca9000640 1 --2- 172.21.15.159:0/4208906398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca41079c0 0x7fcca4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:16.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.439+0000 7fcca3400640 1 --2- 172.21.15.159:0/4208906398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcca4102360 0x7fcca4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:16.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.440+0000 7fcca3e00640 1 -- 172.21.15.159:0/4208906398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca41079c0 msgr2=0x7fcca4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:16.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.440+0000 7fcca3e00640 1 --2- 172.21.15.159:0/4208906398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca41079c0 0x7fcca4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:16.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.440+0000 7fcca3e00640 1 -- 172.21.15.159:0/4208906398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcca4102360 msgr2=0x7fcca4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:16.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.440+0000 7fcca3e00640 1 --2- 172.21.15.159:0/4208906398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcca4102360 0x7fcca4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:16.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.440+0000 7fcca3e00640 1 -- 172.21.15.159:0/4208906398 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca41a6730 con 0x7fcca41019c0 2024-09-19T07:31:16.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.440+0000 7fcca3e00640 1 --2- 172.21.15.159:0/4208906398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcca41019c0 0x7fcca40744f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcc9402a940 tx=0x7fcc94003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:16.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.440+0000 7fcca0c00640 1 -- 172.21.15.159:0/4208906398 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9403f070 con 0x7fcca41019c0 2024-09-19T07:31:16.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.440+0000 7fcca0c00640 1 -- 172.21.15.159:0/4208906398 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc94031c90 con 0x7fcca41019c0 2024-09-19T07:31:16.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.440+0000 7fcca9a00640 1 -- 172.21.15.159:0/4208906398 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca41a69c0 con 0x7fcca41019c0 2024-09-19T07:31:16.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.440+0000 7fcca0c00640 1 -- 172.21.15.159:0/4208906398 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94035030 con 0x7fcca41019c0 2024-09-19T07:31:16.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.440+0000 7fcca9a00640 1 -- 172.21.15.159:0/4208906398 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca41a6ea0 con 0x7fcca41019c0 2024-09-19T07:31:16.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.441+0000 7fcca9a00640 1 -- 172.21.15.159:0/4208906398 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca4101e20 con 0x7fcca41019c0 2024-09-19T07:31:16.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.443+0000 7fcca0c00640 1 -- 172.21.15.159:0/4208906398 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fcc94042070 con 0x7fcca41019c0 2024-09-19T07:31:16.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.444+0000 7fcca0c00640 1 --2- 172.21.15.159:0/4208906398 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcc880a3240 0x7fcc880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:16.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.445+0000 7fcca0c00640 1 -- 172.21.15.159:0/4208906398 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8770+0+0 (secure 0 0 0) 0x7fcc940f0260 con 0x7fcca41019c0 2024-09-19T07:31:16.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.445+0000 7fcca3400640 1 --2- 172.21.15.159:0/4208906398 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcc880a3240 0x7fcc880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:16.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.445+0000 7fcca3400640 1 --2- 172.21.15.159:0/4208906398 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcc880a3240 0x7fcc880a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcca4075f80 tx=0x7fcc9000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:16.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.449+0000 7fcca0c00640 1 -- 172.21.15.159:0/4208906398 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcc940bc6b0 con 0x7fcca41019c0 2024-09-19T07:31:16.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.661+0000 7fcca9a00640 1 -- 172.21.15.159:0/4208906398 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fcca41099a0 con 0x7fcca41019c0 2024-09-19T07:31:16.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.713+0000 7fcca0c00640 1 -- 172.21.15.159:0/4208906398 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fcc940c09b0 con 0x7fcca41019c0 2024-09-19T07:31:16.714 INFO:teuthology.orchestra.run.smithi159.stderr:added data pool 4 to fsmap 2024-09-19T07:31:16.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.716+0000 7fcca9a00640 1 -- 172.21.15.159:0/4208906398 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcc880a3240 msgr2=0x7fcc880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:16.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.716+0000 7fcca9a00640 1 --2- 172.21.15.159:0/4208906398 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcc880a3240 0x7fcc880a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcca4075f80 tx=0x7fcc9000b410 comp rx=0 tx=0).stop 2024-09-19T07:31:16.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.716+0000 7fcca9a00640 1 -- 172.21.15.159:0/4208906398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcca41019c0 msgr2=0x7fcca40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:16.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.716+0000 7fcca9a00640 1 --2- 172.21.15.159:0/4208906398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcca41019c0 0x7fcca40744f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcc9402a940 tx=0x7fcc94003b90 comp rx=0 tx=0).stop 2024-09-19T07:31:16.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.717+0000 7fcca9a00640 1 -- 172.21.15.159:0/4208906398 shutdown_connections 2024-09-19T07:31:16.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.717+0000 7fcca9a00640 1 --2- 172.21.15.159:0/4208906398 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcc880a3240 0x7fcc880a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:16.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.717+0000 7fcca9a00640 1 --2- 172.21.15.159:0/4208906398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca41079c0 0x7fcca4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:16.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.717+0000 7fcca9a00640 1 --2- 172.21.15.159:0/4208906398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcca4102360 0x7fcca4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:16.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.717+0000 7fcca9a00640 1 --2- 172.21.15.159:0/4208906398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcca41019c0 0x7fcca40744f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:16.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.717+0000 7fcca9a00640 1 -- 172.21.15.159:0/4208906398 >> 172.21.15.159:0/4208906398 conn(0x7fcca40fd6e0 msgr2=0x7fcca40feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:16.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.717+0000 7fcca9a00640 1 -- 172.21.15.159:0/4208906398 shutdown_connections 2024-09-19T07:31:16.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:16.717+0000 7fcca9a00640 1 -- 172.21.15.159:0/4208906398 wait complete. 2024-09-19T07:31:17.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:16 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/4208906398' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-19T07:31:17.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:16 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/4208906398' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-19T07:31:17.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:16 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/4208906398' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-19T07:31:17.626 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:31:17.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:17 smithi159 ceph-mon[30472]: pgmap v318: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:17.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:17 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/4208906398' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-19T07:31:17.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:17 smithi159 ceph-mon[30472]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T07:31:17.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:17 smithi159 ceph-mon[30472]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:31:17.999 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:18.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:17 smithi203 ceph-mon[35335]: pgmap v318: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:18.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:17 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/4208906398' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-19T07:31:18.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:17 smithi203 ceph-mon[35335]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T07:31:18.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:17 smithi203 ceph-mon[35335]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:31:18.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:17 smithi189 ceph-mon[34851]: pgmap v318: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:18.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:17 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/4208906398' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-19T07:31:18.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:17 smithi189 ceph-mon[34851]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T07:31:18.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:17 smithi189 ceph-mon[34851]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:31:18.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.619+0000 7f6c8d200640 1 -- 172.21.15.159:0/31541215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c881027a0 msgr2=0x7f6c881067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:18.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.619+0000 7f6c8d200640 1 --2- 172.21.15.159:0/31541215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c881027a0 0x7f6c881067f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009900 tx=0x7f6c7402a3f0 comp rx=0 tx=0).stop 2024-09-19T07:31:18.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.619+0000 7f6c8d200640 1 -- 172.21.15.159:0/31541215 shutdown_connections 2024-09-19T07:31:18.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.619+0000 7f6c8d200640 1 --2- 172.21.15.159:0/31541215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c88106f20 0x7f6c8810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:18.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.619+0000 7f6c8d200640 1 --2- 172.21.15.159:0/31541215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c881027a0 0x7f6c881067f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:18.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.619+0000 7f6c8d200640 1 --2- 172.21.15.159:0/31541215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c88101df0 0x7f6c881021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:18.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.619+0000 7f6c8d200640 1 -- 172.21.15.159:0/31541215 >> 172.21.15.159:0/31541215 conn(0x7f6c880fd8b0 msgr2=0x7f6c880ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:18.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.619+0000 7f6c8d200640 1 -- 172.21.15.159:0/31541215 shutdown_connections 2024-09-19T07:31:18.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.619+0000 7f6c8d200640 1 -- 172.21.15.159:0/31541215 wait complete. 2024-09-19T07:31:18.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.620+0000 7f6c8d200640 1 Processor -- start 2024-09-19T07:31:18.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.620+0000 7f6c8d200640 1 -- start start 2024-09-19T07:31:18.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.621+0000 7f6c8d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c88101df0 0x7f6c8819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:18.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.621+0000 7f6c8d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c881027a0 0x7f6c8819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:18.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.621+0000 7f6c8d200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c88106f20 0x7f6c8819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:18.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.621+0000 7f6c8d200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88111f80 con 0x7f6c881027a0 2024-09-19T07:31:18.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.621+0000 7f6c8d200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88111e00 con 0x7f6c88101df0 2024-09-19T07:31:18.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.621+0000 7f6c8d200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88112100 con 0x7f6c88106f20 2024-09-19T07:31:18.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.621+0000 7f6c85400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c881027a0 0x7f6c8819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:18.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.621+0000 7f6c85400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c881027a0 0x7f6c8819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48878/0 (socket says 172.21.15.159:48878) 2024-09-19T07:31:18.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.621+0000 7f6c85400640 1 -- 172.21.15.159:0/310249549 learned_addr learned my addr 172.21.15.159:0/310249549 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:18.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.621+0000 7f6c85e00640 1 --2- 172.21.15.159:0/310249549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c88101df0 0x7f6c8819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:18.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.621+0000 7f6c86800640 1 --2- 172.21.15.159:0/310249549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c88106f20 0x7f6c8819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:18.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.621+0000 7f6c85400640 1 -- 172.21.15.159:0/310249549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c88106f20 msgr2=0x7f6c8819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:18.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.621+0000 7f6c85400640 1 --2- 172.21.15.159:0/310249549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c88106f20 0x7f6c8819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:18.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.621+0000 7f6c85400640 1 -- 172.21.15.159:0/310249549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c88101df0 msgr2=0x7f6c8819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:18.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.621+0000 7f6c85400640 1 --2- 172.21.15.159:0/310249549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c88101df0 0x7f6c8819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:18.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.621+0000 7f6c85400640 1 -- 172.21.15.159:0/310249549 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8819f5b0 con 0x7f6c881027a0 2024-09-19T07:31:18.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.622+0000 7f6c85400640 1 --2- 172.21.15.159:0/310249549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c881027a0 0x7f6c8819ab40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c74006010 tx=0x7f6c74004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:18.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.622+0000 7f6c7ea00640 1 -- 172.21.15.159:0/310249549 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7403f030 con 0x7f6c881027a0 2024-09-19T07:31:18.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.622+0000 7f6c7ea00640 1 -- 172.21.15.159:0/310249549 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c74004440 con 0x7f6c881027a0 2024-09-19T07:31:18.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.622+0000 7f6c7ea00640 1 -- 172.21.15.159:0/310249549 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74035070 con 0x7f6c881027a0 2024-09-19T07:31:18.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.622+0000 7f6c8d200640 1 -- 172.21.15.159:0/310249549 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8819f840 con 0x7f6c881027a0 2024-09-19T07:31:18.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.622+0000 7f6c8d200640 1 -- 172.21.15.159:0/310249549 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c8819fcd0 con 0x7f6c881027a0 2024-09-19T07:31:18.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.623+0000 7f6c8d200640 1 -- 172.21.15.159:0/310249549 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c881021d0 con 0x7f6c881027a0 2024-09-19T07:31:18.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.625+0000 7f6c7ea00640 1 -- 172.21.15.159:0/310249549 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f6c74042030 con 0x7f6c881027a0 2024-09-19T07:31:18.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.627+0000 7f6c7ea00640 1 --2- 172.21.15.159:0/310249549 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6c6c0a32c0 0x7f6c6c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:18.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.627+0000 7f6c7ea00640 1 -- 172.21.15.159:0/310249549 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f6c740efe70 con 0x7f6c881027a0 2024-09-19T07:31:18.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.627+0000 7f6c85e00640 1 --2- 172.21.15.159:0/310249549 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6c6c0a32c0 0x7f6c6c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:18.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.627+0000 7f6c85e00640 1 --2- 172.21.15.159:0/310249549 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6c6c0a32c0 0x7f6c6c0a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009bd0 tx=0x7f6c78005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:18.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.631+0000 7f6c7ea00640 1 -- 172.21.15.159:0/310249549 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6c740bc2a0 con 0x7f6c881027a0 2024-09-19T07:31:18.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.846+0000 7f6c8d200640 1 -- 172.21.15.159:0/310249549 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c881a7340 con 0x7f6c881027a0 2024-09-19T07:31:18.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.847+0000 7f6c7ea00640 1 -- 172.21.15.159:0/310249549 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11863 (secure 0 0 0) 0x7f6c740c05a0 con 0x7f6c881027a0 2024-09-19T07:31:18.848 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:31:18.848 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":17,"btime":"2024-09-19T07:31:16:713261+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3464449771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3464449771},{"type":"v1","addr":"172.21.15.159:6837","nonce":3464449771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2036264856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2036264856},{"type":"v1","addr":"172.21.15.159:6839","nonce":2036264856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6841/2874566405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6840","nonce":2874566405},{"type":"v1","addr":"172.21.15.159:6841","nonce":2874566405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1340279549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1340279549},{"type":"v1","addr":"172.21.15.189:6835","nonce":1340279549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/937970878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":937970878},{"type":"v1","addr":"172.21.15.203:6835","nonce":937970878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1053443810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1053443810},{"type":"v1","addr":"172.21.15.203:6837","nonce":1053443810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1030825585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1030825585},{"type":"v1","addr":"172.21.15.189:6837","nonce":1030825585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/3862781462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":3862781462},{"type":"v1","addr":"172.21.15.203:6839","nonce":3862781462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:31:03.518840+0000","modified":"2024-09-19T07:31:16.663062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.189:6839/2361818663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2361818663},{"type":"v1","addr":"172.21.15.189:6839","nonce":2361818663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24542,"qdb_cluster":[24542]},"id":1}]} 2024-09-19T07:31:18.848 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 17 2024-09-19T07:31:18.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.849+0000 7f6c8d200640 1 -- 172.21.15.159:0/310249549 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6c6c0a32c0 msgr2=0x7f6c6c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:18.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.849+0000 7f6c8d200640 1 --2- 172.21.15.159:0/310249549 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6c6c0a32c0 0x7f6c6c0a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009bd0 tx=0x7f6c78005f90 comp rx=0 tx=0).stop 2024-09-19T07:31:18.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.849+0000 7f6c8d200640 1 -- 172.21.15.159:0/310249549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c881027a0 msgr2=0x7f6c8819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:18.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.849+0000 7f6c8d200640 1 --2- 172.21.15.159:0/310249549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c881027a0 0x7f6c8819ab40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c74006010 tx=0x7f6c74004040 comp rx=0 tx=0).stop 2024-09-19T07:31:18.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.849+0000 7f6c8d200640 1 -- 172.21.15.159:0/310249549 shutdown_connections 2024-09-19T07:31:18.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.849+0000 7f6c8d200640 1 --2- 172.21.15.159:0/310249549 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6c6c0a32c0 0x7f6c6c0a5780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:18.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.849+0000 7f6c8d200640 1 --2- 172.21.15.159:0/310249549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c88106f20 0x7f6c8819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:18.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.849+0000 7f6c8d200640 1 --2- 172.21.15.159:0/310249549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c881027a0 0x7f6c8819ab40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:18.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.849+0000 7f6c8d200640 1 --2- 172.21.15.159:0/310249549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c88101df0 0x7f6c8819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:18.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.849+0000 7f6c8d200640 1 -- 172.21.15.159:0/310249549 >> 172.21.15.159:0/310249549 conn(0x7f6c880fd8b0 msgr2=0x7f6c880ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:18.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.850+0000 7f6c8d200640 1 -- 172.21.15.159:0/310249549 shutdown_connections 2024-09-19T07:31:18.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:18.850+0000 7f6c8d200640 1 -- 172.21.15.159:0/310249549 wait complete. 2024-09-19T07:31:19.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:18 smithi203 ceph-mon[35335]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-19T07:31:19.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:18 smithi203 ceph-mon[35335]: Cluster is now healthy 2024-09-19T07:31:19.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:18 smithi189 ceph-mon[34851]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-19T07:31:19.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:18 smithi189 ceph-mon[34851]: Cluster is now healthy 2024-09-19T07:31:19.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:18 smithi159 ceph-mon[30472]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-19T07:31:19.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:18 smithi159 ceph-mon[30472]: Cluster is now healthy 2024-09-19T07:31:19.797 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:31:20.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:19 smithi203 ceph-mon[35335]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:20.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:19 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/310249549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:31:20.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:19 smithi189 ceph-mon[34851]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:20.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:19 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/310249549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:31:20.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:19 smithi159 ceph-mon[30472]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:20.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:19 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/310249549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:31:20.168 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:20.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.752+0000 7fae69400640 1 -- 172.21.15.159:0/283649461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae641027a0 msgr2=0x7fae641067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:20.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.752+0000 7fae69400640 1 --2- 172.21.15.159:0/283649461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae641027a0 0x7fae641067f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae50009900 tx=0x7fae5002a400 comp rx=0 tx=0).stop 2024-09-19T07:31:20.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.752+0000 7fae69400640 1 -- 172.21.15.159:0/283649461 shutdown_connections 2024-09-19T07:31:20.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.752+0000 7fae69400640 1 --2- 172.21.15.159:0/283649461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae64106f20 0x7fae6410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:20.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.752+0000 7fae69400640 1 --2- 172.21.15.159:0/283649461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae641027a0 0x7fae641067f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:20.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.752+0000 7fae69400640 1 --2- 172.21.15.159:0/283649461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae64101df0 0x7fae641021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:20.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.752+0000 7fae69400640 1 -- 172.21.15.159:0/283649461 >> 172.21.15.159:0/283649461 conn(0x7fae640fd8b0 msgr2=0x7fae640ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:20.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.752+0000 7fae69400640 1 -- 172.21.15.159:0/283649461 shutdown_connections 2024-09-19T07:31:20.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.752+0000 7fae69400640 1 -- 172.21.15.159:0/283649461 wait complete. 2024-09-19T07:31:20.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.753+0000 7fae69400640 1 Processor -- start 2024-09-19T07:31:20.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.753+0000 7fae69400640 1 -- start start 2024-09-19T07:31:20.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.754+0000 7fae69400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae64101df0 0x7fae6419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:20.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.754+0000 7fae69400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae641027a0 0x7fae6419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:20.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.754+0000 7fae69400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae64106f20 0x7fae6419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:20.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.754+0000 7fae69400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fae64111f80 con 0x7fae64106f20 2024-09-19T07:31:20.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.754+0000 7fae69400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae64111e00 con 0x7fae641027a0 2024-09-19T07:31:20.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.754+0000 7fae69400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fae64112100 con 0x7fae64101df0 2024-09-19T07:31:20.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.754+0000 7fae62800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae64101df0 0x7fae6419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:20.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.754+0000 7fae62800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae64101df0 0x7fae6419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:49566/0 (socket says 172.21.15.159:49566) 2024-09-19T07:31:20.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.754+0000 7fae62800640 1 -- 172.21.15.159:0/1842845633 learned_addr learned my addr 172.21.15.159:0/1842845633 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:20.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.754+0000 7fae61e00640 1 --2- 172.21.15.159:0/1842845633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae641027a0 0x7fae6419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:20.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.754+0000 7fae63200640 1 --2- 172.21.15.159:0/1842845633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae64106f20 0x7fae6419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:20.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.754+0000 7fae62800640 1 -- 172.21.15.159:0/1842845633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae641027a0 msgr2=0x7fae6419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:20.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.754+0000 7fae62800640 1 --2- 172.21.15.159:0/1842845633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae641027a0 0x7fae6419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:20.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.754+0000 7fae62800640 1 -- 172.21.15.159:0/1842845633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae64106f20 msgr2=0x7fae6419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:20.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.754+0000 7fae62800640 1 --2- 172.21.15.159:0/1842845633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae64106f20 0x7fae6419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:20.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.754+0000 7fae62800640 1 -- 172.21.15.159:0/1842845633 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6419f600 con 0x7fae64101df0 2024-09-19T07:31:20.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.754+0000 7fae63200640 1 --2- 172.21.15.159:0/1842845633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae64106f20 0x7fae6419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:31:20.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.755+0000 7fae62800640 1 --2- 172.21.15.159:0/1842845633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae64101df0 0x7fae6419a650 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae5400e9f0 tx=0x7fae5400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:20.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.755+0000 7fae5b400640 1 -- 172.21.15.159:0/1842845633 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5400cd80 con 0x7fae64101df0 2024-09-19T07:31:20.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.755+0000 7fae69400640 1 -- 172.21.15.159:0/1842845633 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6419f8f0 con 0x7fae64101df0 2024-09-19T07:31:20.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.755+0000 7fae5b400640 1 -- 172.21.15.159:0/1842845633 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae54010910 con 0x7fae64101df0 2024-09-19T07:31:20.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.755+0000 7fae5b400640 1 -- 172.21.15.159:0/1842845633 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54018990 con 0x7fae64101df0 2024-09-19T07:31:20.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.755+0000 7fae69400640 1 -- 172.21.15.159:0/1842845633 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae641a71d0 con 0x7fae64101df0 2024-09-19T07:31:20.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.756+0000 7fae69400640 1 -- 172.21.15.159:0/1842845633 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae641021d0 con 0x7fae64101df0 2024-09-19T07:31:20.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.758+0000 7fae5b400640 1 -- 172.21.15.159:0/1842845633 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fae54010ac0 con 0x7fae64101df0 2024-09-19T07:31:20.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.759+0000 7fae5b400640 1 --2- 172.21.15.159:0/1842845633 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fae480a3240 0x7fae480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:20.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.760+0000 7fae5b400640 1 -- 172.21.15.159:0/1842845633 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7fae540d03b0 con 0x7fae64101df0 2024-09-19T07:31:20.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.760+0000 7fae61e00640 1 --2- 172.21.15.159:0/1842845633 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fae480a3240 0x7fae480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:20.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.760+0000 7fae61e00640 1 --2- 172.21.15.159:0/1842845633 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fae480a3240 0x7fae480a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fae50009230 tx=0x7fae500058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:20.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.765+0000 7fae5b400640 1 -- 172.21.15.159:0/1842845633 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fae5409ce30 con 0x7fae64101df0 2024-09-19T07:31:20.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.945+0000 7fae69400640 1 -- 172.21.15.159:0/1842845633 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae641a73d0 con 0x7fae64101df0 2024-09-19T07:31:20.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.947+0000 7fae5b400640 1 -- 172.21.15.159:0/1842845633 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24294 (secure 0 0 0) 0x7fae5409c730 con 0x7fae64101df0 2024-09-19T07:31:20.949 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:31:20.949 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":96,"fsid":"c9d18048-7657-11ef-bceb-c7b262605968","created":"2024-09-19T07:22:32.059836+0000","modified":"2024-09-19T07:31:16.662906+0000","last_up_change":"2024-09-19T07:30:03.328697+0000","last_in_change":"2024-09-19T07:29:38.987579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:25:55.202457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:30:58.022056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:31:01.042290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:31:10.977289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"752f5466-fe04-408a-8566-86625947fd3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6803","nonce":2738900830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6805","nonce":2738900830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6809","nonce":2738900830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6807","nonce":2738900830}]},"public_addr":"172.21.15.159:6803/2738900830","cluster_addr":"172.21.15.159:6805/2738900830","heartbeat_back_addr":"172.21.15.159:6809/2738900830","heartbeat_front_addr":"172.21.15.159:6807/2738900830","state":["exists","up"]},{"osd":1,"uuid":"7513adda-ca82-4df4-aa99-3b050b410424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6801","nonce":265696178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6803","nonce":265696178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6807","nonce":265696178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6805","nonce":265696178}]},"public_addr":"172.21.15.189:6801/265696178","cluster_addr":"172.21.15.189:6803/265696178","heartbeat_back_addr":"172.21.15.189:6807/265696178","heartbeat_front_addr":"172.21.15.189:6805/265696178","state":["exists","up"]},{"osd":2,"uuid":"aa03be79-58d8-4eb7-a051-0b46c502b12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6801","nonce":1826565288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6803","nonce":1826565288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6807","nonce":1826565288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6805","nonce":1826565288}]},"public_addr":"172.21.15.203:6801/1826565288","cluster_addr":"172.21.15.203:6803/1826565288","heartbeat_back_addr":"172.21.15.203:6807/1826565288","heartbeat_front_addr":"172.21.15.203:6805/1826565288","state":["exists","up"]},{"osd":3,"uuid":"fcd734f1-f40f-4566-82ab-c4607039f23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6811","nonce":109457380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6813","nonce":109457380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6817","nonce":109457380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6815","nonce":109457380}]},"public_addr":"172.21.15.159:6811/109457380","cluster_addr":"172.21.15.159:6813/109457380","heartbeat_back_addr":"172.21.15.159:6817/109457380","heartbeat_front_addr":"172.21.15.159:6815/109457380","state":["exists","up"]},{"osd":4,"uuid":"bc344c9e-2267-4058-bbe1-9c270327db8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6809","nonce":1036888180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6811","nonce":1036888180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6815","nonce":1036888180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6813","nonce":1036888180}]},"public_addr":"172.21.15.189:6809/1036888180","cluster_addr":"172.21.15.189:6811/1036888180","heartbeat_back_addr":"172.21.15.189:6815/1036888180","heartbeat_front_addr":"172.21.15.189:6813/1036888180","state":["exists","up"]},{"osd":5,"uuid":"86c48303-929d-42f4-bf48-cbb32649f95f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6809","nonce":4212601512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6811","nonce":4212601512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6815","nonce":4212601512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6813","nonce":4212601512}]},"public_addr":"172.21.15.203:6809/4212601512","cluster_addr":"172.21.15.203:6811/4212601512","heartbeat_back_addr":"172.21.15.203:6815/4212601512","heartbeat_front_addr":"172.21.15.203:6813/4212601512","state":["exists","up"]},{"osd":6,"uuid":"f0176ed1-c3a7-4d75-b213-fe88fab4441a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6819","nonce":3817684578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6821","nonce":3817684578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6825","nonce":3817684578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6823","nonce":3817684578}]},"public_addr":"172.21.15.159:6819/3817684578","cluster_addr":"172.21.15.159:6821/3817684578","heartbeat_back_addr":"172.21.15.159:6825/3817684578","heartbeat_front_addr":"172.21.15.159:6823/3817684578","state":["exists","up"]},{"osd":7,"uuid":"921bd59d-9415-4559-8e9f-035d336afd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6817","nonce":3936826456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6819","nonce":3936826456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6823","nonce":3936826456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6821","nonce":3936826456}]},"public_addr":"172.21.15.189:6817/3936826456","cluster_addr":"172.21.15.189:6819/3936826456","heartbeat_back_addr":"172.21.15.189:6823/3936826456","heartbeat_front_addr":"172.21.15.189:6821/3936826456","state":["exists","up"]},{"osd":8,"uuid":"ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6817","nonce":4062926645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6819","nonce":4062926645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6823","nonce":4062926645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6821","nonce":4062926645}]},"public_addr":"172.21.15.203:6817/4062926645","cluster_addr":"172.21.15.203:6819/4062926645","heartbeat_back_addr":"172.21.15.203:6823/4062926645","heartbeat_front_addr":"172.21.15.203:6821/4062926645","state":["exists","up"]},{"osd":9,"uuid":"df29a10f-1c60-41cb-9c2d-bc9f1cb174bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6827","nonce":4185848747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6829","nonce":4185848747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6833","nonce":4185848747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6831","nonce":4185848747}]},"public_addr":"172.21.15.159:6827/4185848747","cluster_addr":"172.21.15.159:6829/4185848747","heartbeat_back_addr":"172.21.15.159:6833/4185848747","heartbeat_front_addr":"172.21.15.159:6831/4185848747","state":["exists","up"]},{"osd":10,"uuid":"ccc440b4-4682-4998-bb53-95e24423cd1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6825","nonce":3018377605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6827","nonce":3018377605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6831","nonce":3018377605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6829","nonce":3018377605}]},"public_addr":"172.21.15.189:6825/3018377605","cluster_addr":"172.21.15.189:6827/3018377605","heartbeat_back_addr":"172.21.15.189:6831/3018377605","heartbeat_front_addr":"172.21.15.189:6829/3018377605","state":["exists","up"]},{"osd":11,"uuid":"e84afc5f-8051-4341-8fd0-3afb482ab4df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6825","nonce":2807503891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6827","nonce":2807503891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6831","nonce":2807503891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6829","nonce":2807503891}]},"public_addr":"172.21.15.203:6825/2807503891","cluster_addr":"172.21.15.203:6827/2807503891","heartbeat_back_addr":"172.21.15.203:6831/2807503891","heartbeat_front_addr":"172.21.15.203:6829/2807503891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:00.093677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:25.477205+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:51.345020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:21.077603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:48.822640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:13.516029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:44.382228+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:10.810700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:36.817155+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:07.728553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:34.747540+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:01.249332+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:0/2383222735":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1561371050":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6800/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:6800/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/2108730247":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/3393992615":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1366863724":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/1628344860":"2024-09-20T07:23:03.089190+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:31:20.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.950+0000 7fae69400640 1 -- 172.21.15.159:0/1842845633 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fae480a3240 msgr2=0x7fae480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:20.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.950+0000 7fae69400640 1 --2- 172.21.15.159:0/1842845633 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fae480a3240 0x7fae480a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fae50009230 tx=0x7fae500058d0 comp rx=0 tx=0).stop 2024-09-19T07:31:20.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.950+0000 7fae69400640 1 -- 172.21.15.159:0/1842845633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae64101df0 msgr2=0x7fae6419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:20.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.950+0000 7fae69400640 1 --2- 172.21.15.159:0/1842845633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae64101df0 0x7fae6419a650 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae5400e9f0 tx=0x7fae5400eeb0 comp rx=0 tx=0).stop 2024-09-19T07:31:20.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.950+0000 7fae69400640 1 -- 172.21.15.159:0/1842845633 shutdown_connections 2024-09-19T07:31:20.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.950+0000 7fae69400640 1 --2- 172.21.15.159:0/1842845633 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fae480a3240 0x7fae480a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:20.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.950+0000 7fae69400640 1 --2- 172.21.15.159:0/1842845633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae64106f20 0x7fae6419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:20.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.950+0000 7fae69400640 1 --2- 172.21.15.159:0/1842845633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae641027a0 0x7fae6419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:20.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.950+0000 7fae69400640 1 --2- 172.21.15.159:0/1842845633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae64101df0 0x7fae6419a650 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:20.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.950+0000 7fae69400640 1 -- 172.21.15.159:0/1842845633 >> 172.21.15.159:0/1842845633 conn(0x7fae640fd8b0 msgr2=0x7fae640fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:20.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.950+0000 7fae69400640 1 -- 172.21.15.159:0/1842845633 shutdown_connections 2024-09-19T07:31:20.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:20.951+0000 7fae69400640 1 -- 172.21.15.159:0/1842845633 wait complete. 2024-09-19T07:31:21.883 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:31:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:21 smithi203 ceph-mon[35335]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:22.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:21 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1842845633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:31:22.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:21 smithi189 ceph-mon[34851]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:22.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:21 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1842845633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:31:22.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:21 smithi159 ceph-mon[30472]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:22.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:21 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1842845633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:31:22.251 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:22.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.865+0000 7f1889800640 1 -- 172.21.15.159:0/2415723217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1884103510 msgr2=0x7f188410ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:22.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.865+0000 7f1889800640 1 --2- 172.21.15.159:0/2415723217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1884103510 0x7f188410ba00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1870009900 tx=0x7f187002a480 comp rx=0 tx=0).stop 2024-09-19T07:31:22.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.865+0000 7f1889800640 1 -- 172.21.15.159:0/2415723217 shutdown_connections 2024-09-19T07:31:22.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.865+0000 7f1889800640 1 --2- 172.21.15.159:0/2415723217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f188410bfd0 0x7f188410e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:22.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.865+0000 7f1889800640 1 --2- 172.21.15.159:0/2415723217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1884103510 0x7f188410ba00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:22.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.865+0000 7f1889800640 1 --2- 172.21.15.159:0/2415723217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1884102bf0 0x7f1884102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:22.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.865+0000 7f1889800640 1 -- 172.21.15.159:0/2415723217 >> 172.21.15.159:0/2415723217 conn(0x7f18840fc610 msgr2=0x7f18840fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:22.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.865+0000 7f1889800640 1 -- 172.21.15.159:0/2415723217 shutdown_connections 2024-09-19T07:31:22.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.865+0000 7f1889800640 1 -- 172.21.15.159:0/2415723217 wait complete. 2024-09-19T07:31:22.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.866+0000 7f1889800640 1 Processor -- start 2024-09-19T07:31:22.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.866+0000 7f1889800640 1 -- start start 2024-09-19T07:31:22.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.866+0000 7f1889800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1884102bf0 0x7f18841982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:22.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.866+0000 7f1889800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1884103510 0x7f18841987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:22.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.867+0000 7f1889800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f188410bfd0 0x7f188419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.867+0000 7f1889800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1884111db0 con 0x7f1884102bf0 2024-09-19T07:31:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.867+0000 7f1889800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1884111c30 con 0x7f1884103510 2024-09-19T07:31:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.867+0000 7f1889800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1884111f30 con 0x7f188410bfd0 2024-09-19T07:31:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.867+0000 7f1883e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1884102bf0 0x7f18841982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.867+0000 7f1883e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1884102bf0 0x7f18841982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59802/0 (socket says 172.21.15.159:59802) 2024-09-19T07:31:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.867+0000 7f1883e00640 1 -- 172.21.15.159:0/341995256 learned_addr learned my addr 172.21.15.159:0/341995256 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.867+0000 7f1883400640 1 --2- 172.21.15.159:0/341995256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1884103510 0x7f18841987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.867+0000 7f1888e00640 1 --2- 172.21.15.159:0/341995256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f188410bfd0 0x7f188419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.867+0000 7f1883e00640 1 -- 172.21.15.159:0/341995256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f188410bfd0 msgr2=0x7f188419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.867+0000 7f1883e00640 1 --2- 172.21.15.159:0/341995256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f188410bfd0 0x7f188419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.867+0000 7f1883e00640 1 -- 172.21.15.159:0/341995256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1884103510 msgr2=0x7f18841987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.867+0000 7f1883e00640 1 --2- 172.21.15.159:0/341995256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1884103510 0x7f18841987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.867+0000 7f1883e00640 1 -- 172.21.15.159:0/341995256 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188419d260 con 0x7f1884102bf0 2024-09-19T07:31:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.868+0000 7f1883e00640 1 --2- 172.21.15.159:0/341995256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1884102bf0 0x7f18841982b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f187400d770 tx=0x7f187400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:22.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.868+0000 7f1880c00640 1 -- 172.21.15.159:0/341995256 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874017070 con 0x7f1884102bf0 2024-09-19T07:31:22.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.868+0000 7f1880c00640 1 -- 172.21.15.159:0/341995256 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1874005500 con 0x7f1884102bf0 2024-09-19T07:31:22.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.868+0000 7f1889800640 1 -- 172.21.15.159:0/341995256 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188419d550 con 0x7f1884102bf0 2024-09-19T07:31:22.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.868+0000 7f1880c00640 1 -- 172.21.15.159:0/341995256 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874004e00 con 0x7f1884102bf0 2024-09-19T07:31:22.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.868+0000 7f1889800640 1 -- 172.21.15.159:0/341995256 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18841a4e30 con 0x7f1884102bf0 2024-09-19T07:31:22.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.869+0000 7f1889800640 1 -- 172.21.15.159:0/341995256 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f188406a7f0 con 0x7f1884102bf0 2024-09-19T07:31:22.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.871+0000 7f1880c00640 1 -- 172.21.15.159:0/341995256 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f1874004fa0 con 0x7f1884102bf0 2024-09-19T07:31:22.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.872+0000 7f1880c00640 1 --2- 172.21.15.159:0/341995256 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f18680a3240 0x7f18680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:22.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.872+0000 7f1880c00640 1 -- 172.21.15.159:0/341995256 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f18740cf590 con 0x7f1884102bf0 2024-09-19T07:31:22.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.873+0000 7f1883400640 1 --2- 172.21.15.159:0/341995256 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f18680a3240 0x7f18680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:22.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.873+0000 7f1883400640 1 --2- 172.21.15.159:0/341995256 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f18680a3240 0x7f18680a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1870009230 tx=0x7f18700058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:22.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:22.876+0000 7f1880c00640 1 -- 172.21.15.159:0/341995256 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f187409b9c0 con 0x7f1884102bf0 2024-09-19T07:31:23.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:23.061+0000 7f1889800640 1 -- 172.21.15.159:0/341995256 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1884107010 con 0x7f1884102bf0 2024-09-19T07:31:23.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:23.063+0000 7f1880c00640 1 -- 172.21.15.159:0/341995256 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24294 (secure 0 0 0) 0x7f187409fcc0 con 0x7f1884102bf0 2024-09-19T07:31:23.064 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:31:23.065 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":96,"fsid":"c9d18048-7657-11ef-bceb-c7b262605968","created":"2024-09-19T07:22:32.059836+0000","modified":"2024-09-19T07:31:16.662906+0000","last_up_change":"2024-09-19T07:30:03.328697+0000","last_in_change":"2024-09-19T07:29:38.987579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:25:55.202457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:30:58.022056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:31:01.042290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:31:10.977289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"752f5466-fe04-408a-8566-86625947fd3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6803","nonce":2738900830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6805","nonce":2738900830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6809","nonce":2738900830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6807","nonce":2738900830}]},"public_addr":"172.21.15.159:6803/2738900830","cluster_addr":"172.21.15.159:6805/2738900830","heartbeat_back_addr":"172.21.15.159:6809/2738900830","heartbeat_front_addr":"172.21.15.159:6807/2738900830","state":["exists","up"]},{"osd":1,"uuid":"7513adda-ca82-4df4-aa99-3b050b410424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6801","nonce":265696178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6803","nonce":265696178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6807","nonce":265696178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6805","nonce":265696178}]},"public_addr":"172.21.15.189:6801/265696178","cluster_addr":"172.21.15.189:6803/265696178","heartbeat_back_addr":"172.21.15.189:6807/265696178","heartbeat_front_addr":"172.21.15.189:6805/265696178","state":["exists","up"]},{"osd":2,"uuid":"aa03be79-58d8-4eb7-a051-0b46c502b12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6801","nonce":1826565288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6803","nonce":1826565288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6807","nonce":1826565288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6805","nonce":1826565288}]},"public_addr":"172.21.15.203:6801/1826565288","cluster_addr":"172.21.15.203:6803/1826565288","heartbeat_back_addr":"172.21.15.203:6807/1826565288","heartbeat_front_addr":"172.21.15.203:6805/1826565288","state":["exists","up"]},{"osd":3,"uuid":"fcd734f1-f40f-4566-82ab-c4607039f23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6811","nonce":109457380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6813","nonce":109457380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6817","nonce":109457380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6815","nonce":109457380}]},"public_addr":"172.21.15.159:6811/109457380","cluster_addr":"172.21.15.159:6813/109457380","heartbeat_back_addr":"172.21.15.159:6817/109457380","heartbeat_front_addr":"172.21.15.159:6815/109457380","state":["exists","up"]},{"osd":4,"uuid":"bc344c9e-2267-4058-bbe1-9c270327db8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6809","nonce":1036888180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6811","nonce":1036888180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6815","nonce":1036888180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6813","nonce":1036888180}]},"public_addr":"172.21.15.189:6809/1036888180","cluster_addr":"172.21.15.189:6811/1036888180","heartbeat_back_addr":"172.21.15.189:6815/1036888180","heartbeat_front_addr":"172.21.15.189:6813/1036888180","state":["exists","up"]},{"osd":5,"uuid":"86c48303-929d-42f4-bf48-cbb32649f95f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6809","nonce":4212601512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6811","nonce":4212601512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6815","nonce":4212601512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6813","nonce":4212601512}]},"public_addr":"172.21.15.203:6809/4212601512","cluster_addr":"172.21.15.203:6811/4212601512","heartbeat_back_addr":"172.21.15.203:6815/4212601512","heartbeat_front_addr":"172.21.15.203:6813/4212601512","state":["exists","up"]},{"osd":6,"uuid":"f0176ed1-c3a7-4d75-b213-fe88fab4441a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6819","nonce":3817684578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6821","nonce":3817684578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6825","nonce":3817684578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6823","nonce":3817684578}]},"public_addr":"172.21.15.159:6819/3817684578","cluster_addr":"172.21.15.159:6821/3817684578","heartbeat_back_addr":"172.21.15.159:6825/3817684578","heartbeat_front_addr":"172.21.15.159:6823/3817684578","state":["exists","up"]},{"osd":7,"uuid":"921bd59d-9415-4559-8e9f-035d336afd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6817","nonce":3936826456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6819","nonce":3936826456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6823","nonce":3936826456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6821","nonce":3936826456}]},"public_addr":"172.21.15.189:6817/3936826456","cluster_addr":"172.21.15.189:6819/3936826456","heartbeat_back_addr":"172.21.15.189:6823/3936826456","heartbeat_front_addr":"172.21.15.189:6821/3936826456","state":["exists","up"]},{"osd":8,"uuid":"ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6817","nonce":4062926645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6819","nonce":4062926645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6823","nonce":4062926645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6821","nonce":4062926645}]},"public_addr":"172.21.15.203:6817/4062926645","cluster_addr":"172.21.15.203:6819/4062926645","heartbeat_back_addr":"172.21.15.203:6823/4062926645","heartbeat_front_addr":"172.21.15.203:6821/4062926645","state":["exists","up"]},{"osd":9,"uuid":"df29a10f-1c60-41cb-9c2d-bc9f1cb174bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6827","nonce":4185848747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6829","nonce":4185848747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6833","nonce":4185848747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6831","nonce":4185848747}]},"public_addr":"172.21.15.159:6827/4185848747","cluster_addr":"172.21.15.159:6829/4185848747","heartbeat_back_addr":"172.21.15.159:6833/4185848747","heartbeat_front_addr":"172.21.15.159:6831/4185848747","state":["exists","up"]},{"osd":10,"uuid":"ccc440b4-4682-4998-bb53-95e24423cd1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6825","nonce":3018377605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6827","nonce":3018377605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6831","nonce":3018377605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6829","nonce":3018377605}]},"public_addr":"172.21.15.189:6825/3018377605","cluster_addr":"172.21.15.189:6827/3018377605","heartbeat_back_addr":"172.21.15.189:6831/3018377605","heartbeat_front_addr":"172.21.15.189:6829/3018377605","state":["exists","up"]},{"osd":11,"uuid":"e84afc5f-8051-4341-8fd0-3afb482ab4df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6825","nonce":2807503891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6827","nonce":2807503891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6831","nonce":2807503891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6829","nonce":2807503891}]},"public_addr":"172.21.15.203:6825/2807503891","cluster_addr":"172.21.15.203:6827/2807503891","heartbeat_back_addr":"172.21.15.203:6831/2807503891","heartbeat_front_addr":"172.21.15.203:6829/2807503891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:00.093677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:25.477205+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:51.345020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:21.077603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:48.822640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:13.516029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:44.382228+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:10.810700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:36.817155+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:07.728553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:34.747540+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:01.249332+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:0/2383222735":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1561371050":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6800/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:6800/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/2108730247":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/3393992615":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1366863724":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/1628344860":"2024-09-20T07:23:03.089190+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:31:23.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:23.066+0000 7f1889800640 1 -- 172.21.15.159:0/341995256 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f18680a3240 msgr2=0x7f18680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:23.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:23.066+0000 7f1889800640 1 --2- 172.21.15.159:0/341995256 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f18680a3240 0x7f18680a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1870009230 tx=0x7f18700058d0 comp rx=0 tx=0).stop 2024-09-19T07:31:23.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:23.066+0000 7f1889800640 1 -- 172.21.15.159:0/341995256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1884102bf0 msgr2=0x7f18841982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:23.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:23.066+0000 7f1889800640 1 --2- 172.21.15.159:0/341995256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1884102bf0 0x7f18841982b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f187400d770 tx=0x7f187400dc30 comp rx=0 tx=0).stop 2024-09-19T07:31:23.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:23.066+0000 7f1889800640 1 -- 172.21.15.159:0/341995256 shutdown_connections 2024-09-19T07:31:23.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:23.066+0000 7f1889800640 1 --2- 172.21.15.159:0/341995256 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f18680a3240 0x7f18680a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:23.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:23.066+0000 7f1889800640 1 --2- 172.21.15.159:0/341995256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f188410bfd0 0x7f188419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:23.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:23.066+0000 7f1889800640 1 --2- 172.21.15.159:0/341995256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1884103510 0x7f18841987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:23.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:23.066+0000 7f1889800640 1 --2- 172.21.15.159:0/341995256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1884102bf0 0x7f18841982b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:23.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:23.066+0000 7f1889800640 1 -- 172.21.15.159:0/341995256 >> 172.21.15.159:0/341995256 conn(0x7f18840fc610 msgr2=0x7f18840fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:23.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:23.066+0000 7f1889800640 1 -- 172.21.15.159:0/341995256 shutdown_connections 2024-09-19T07:31:23.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:23.066+0000 7f1889800640 1 -- 172.21.15.159:0/341995256 wait complete. 2024-09-19T07:31:23.881 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:31:24.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:23 smithi203 ceph-mon[35335]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:24.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:23 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/341995256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:31:24.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:23 smithi189 ceph-mon[34851]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:24.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:23 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/341995256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:31:24.104 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:23 smithi159 ceph-mon[30472]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:24.104 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:23 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/341995256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:31:24.260 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.923+0000 7f208de00640 1 -- 172.21.15.159:0/1707975154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2088077040 msgr2=0x7f2088112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.923+0000 7f208de00640 1 --2- 172.21.15.159:0/1707975154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2088077040 0x7f2088112330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2074009900 tx=0x7f207402a410 comp rx=0 tx=0).stop 2024-09-19T07:31:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.923+0000 7f208de00640 1 -- 172.21.15.159:0/1707975154 shutdown_connections 2024-09-19T07:31:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.923+0000 7f208de00640 1 --2- 172.21.15.159:0/1707975154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2088112870 0x7f2088114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.923+0000 7f208de00640 1 --2- 172.21.15.159:0/1707975154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2088077040 0x7f2088112330 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.923+0000 7f208de00640 1 --2- 172.21.15.159:0/1707975154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2088076720 0x7f2088076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.923+0000 7f208de00640 1 -- 172.21.15.159:0/1707975154 >> 172.21.15.159:0/1707975154 conn(0x7f20880ff910 msgr2=0x7f2088101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.923+0000 7f208de00640 1 -- 172.21.15.159:0/1707975154 shutdown_connections 2024-09-19T07:31:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.923+0000 7f208de00640 1 -- 172.21.15.159:0/1707975154 wait complete. 2024-09-19T07:31:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.924+0000 7f208de00640 1 Processor -- start 2024-09-19T07:31:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.924+0000 7f208de00640 1 -- start start 2024-09-19T07:31:24.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.925+0000 7f208de00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2088076720 0x7f20881a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:24.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.925+0000 7f208de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2088077040 0x7f20881a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:24.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.925+0000 7f208de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2088112870 0x7f208819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:24.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.925+0000 7f208de00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f20881183e0 con 0x7f2088112870 2024-09-19T07:31:24.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.925+0000 7f208de00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2088118260 con 0x7f2088077040 2024-09-19T07:31:24.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.925+0000 7f208de00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2088118560 con 0x7f2088076720 2024-09-19T07:31:24.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.925+0000 7f208ca00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2088076720 0x7f20881a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:24.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.925+0000 7f208d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2088112870 0x7f208819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:24.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.925+0000 7f208ca00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2088076720 0x7f20881a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:45270/0 (socket says 172.21.15.159:45270) 2024-09-19T07:31:24.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.925+0000 7f208ca00640 1 -- 172.21.15.159:0/440667217 learned_addr learned my addr 172.21.15.159:0/440667217 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:24.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.925+0000 7f2087e00640 1 --2- 172.21.15.159:0/440667217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2088077040 0x7f20881a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:24.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.925+0000 7f208ca00640 1 -- 172.21.15.159:0/440667217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2088077040 msgr2=0x7f20881a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:24.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.925+0000 7f208ca00640 1 --2- 172.21.15.159:0/440667217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2088077040 0x7f20881a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:24.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.925+0000 7f208ca00640 1 -- 172.21.15.159:0/440667217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2088112870 msgr2=0x7f208819e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:24.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.925+0000 7f208ca00640 1 --2- 172.21.15.159:0/440667217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2088112870 0x7f208819e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:24.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.925+0000 7f208ca00640 1 -- 172.21.15.159:0/440667217 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208819ef50 con 0x7f2088076720 2024-09-19T07:31:24.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.925+0000 7f208d400640 1 --2- 172.21.15.159:0/440667217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2088112870 0x7f208819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:31:24.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.926+0000 7f208ca00640 1 --2- 172.21.15.159:0/440667217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2088076720 0x7f20881a4630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f207c00d950 tx=0x7f207c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:24.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.926+0000 7f2085600640 1 -- 172.21.15.159:0/440667217 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c017070 con 0x7f2088076720 2024-09-19T07:31:24.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.926+0000 7f2085600640 1 -- 172.21.15.159:0/440667217 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f207c009c30 con 0x7f2088076720 2024-09-19T07:31:24.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.926+0000 7f208de00640 1 -- 172.21.15.159:0/440667217 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208819f240 con 0x7f2088076720 2024-09-19T07:31:24.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.926+0000 7f2085600640 1 -- 172.21.15.159:0/440667217 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c00f6b0 con 0x7f2088076720 2024-09-19T07:31:24.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.926+0000 7f208de00640 1 -- 172.21.15.159:0/440667217 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20881ab490 con 0x7f2088076720 2024-09-19T07:31:24.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.927+0000 7f208de00640 1 -- 172.21.15.159:0/440667217 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2088077e60 con 0x7f2088076720 2024-09-19T07:31:24.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.929+0000 7f2085600640 1 -- 172.21.15.159:0/440667217 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f207c0109d0 con 0x7f2088076720 2024-09-19T07:31:24.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.930+0000 7f2085600640 1 --2- 172.21.15.159:0/440667217 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f20700a3240 0x7f20700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:24.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.930+0000 7f2085600640 1 -- 172.21.15.159:0/440667217 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f207c0cf790 con 0x7f2088076720 2024-09-19T07:31:24.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.930+0000 7f2087e00640 1 --2- 172.21.15.159:0/440667217 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f20700a3240 0x7f20700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:24.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.931+0000 7f2087e00640 1 --2- 172.21.15.159:0/440667217 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f20700a3240 0x7f20700a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2074009230 tx=0x7f20740058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:24.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:24.934+0000 7f2085600640 1 -- 172.21.15.159:0/440667217 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f207c009de0 con 0x7f2088076720 2024-09-19T07:31:25.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:25.147+0000 7f208de00640 1 -- 172.21.15.159:0/440667217 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f208810da00 con 0x7f2088076720 2024-09-19T07:31:25.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:25.148+0000 7f2085600640 1 -- 172.21.15.159:0/440667217 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11863 (secure 0 0 0) 0x7f207c09bbc0 con 0x7f2088076720 2024-09-19T07:31:25.149 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:31:25.149 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":17,"btime":"2024-09-19T07:31:16:713261+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3464449771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3464449771},{"type":"v1","addr":"172.21.15.159:6837","nonce":3464449771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2036264856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2036264856},{"type":"v1","addr":"172.21.15.159:6839","nonce":2036264856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6841/2874566405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6840","nonce":2874566405},{"type":"v1","addr":"172.21.15.159:6841","nonce":2874566405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1340279549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1340279549},{"type":"v1","addr":"172.21.15.189:6835","nonce":1340279549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/937970878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":937970878},{"type":"v1","addr":"172.21.15.203:6835","nonce":937970878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1053443810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1053443810},{"type":"v1","addr":"172.21.15.203:6837","nonce":1053443810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1030825585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1030825585},{"type":"v1","addr":"172.21.15.189:6837","nonce":1030825585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/3862781462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":3862781462},{"type":"v1","addr":"172.21.15.203:6839","nonce":3862781462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T07:31:03.518840+0000","modified":"2024-09-19T07:31:16.663062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.189:6839/2361818663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2361818663},{"type":"v1","addr":"172.21.15.189:6839","nonce":2361818663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24542,"qdb_cluster":[24542]},"id":1}]} 2024-09-19T07:31:25.149 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 17 2024-09-19T07:31:25.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:25.150+0000 7f208de00640 1 -- 172.21.15.159:0/440667217 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f20700a3240 msgr2=0x7f20700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:25.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:25.150+0000 7f208de00640 1 --2- 172.21.15.159:0/440667217 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f20700a3240 0x7f20700a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2074009230 tx=0x7f20740058d0 comp rx=0 tx=0).stop 2024-09-19T07:31:25.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:25.150+0000 7f208de00640 1 -- 172.21.15.159:0/440667217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2088076720 msgr2=0x7f20881a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:25.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:25.150+0000 7f208de00640 1 --2- 172.21.15.159:0/440667217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2088076720 0x7f20881a4630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f207c00d950 tx=0x7f207c00de10 comp rx=0 tx=0).stop 2024-09-19T07:31:25.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:25.150+0000 7f208de00640 1 -- 172.21.15.159:0/440667217 shutdown_connections 2024-09-19T07:31:25.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:25.150+0000 7f208de00640 1 --2- 172.21.15.159:0/440667217 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f20700a3240 0x7f20700a5700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:25.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:25.150+0000 7f208de00640 1 --2- 172.21.15.159:0/440667217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2088112870 0x7f208819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:25.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:25.150+0000 7f208de00640 1 --2- 172.21.15.159:0/440667217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2088077040 0x7f20881a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:25.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:25.150+0000 7f208de00640 1 --2- 172.21.15.159:0/440667217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2088076720 0x7f20881a4630 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:25.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:25.150+0000 7f208de00640 1 -- 172.21.15.159:0/440667217 >> 172.21.15.159:0/440667217 conn(0x7f20880ff910 msgr2=0x7f2088101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:25.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:25.151+0000 7f208de00640 1 -- 172.21.15.159:0/440667217 shutdown_connections 2024-09-19T07:31:25.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:25.151+0000 7f208de00640 1 -- 172.21.15.159:0/440667217 wait complete. 2024-09-19T07:31:25.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:25 smithi159 ceph-mon[30472]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:25.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:25 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/440667217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:31:26.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:25 smithi203 ceph-mon[35335]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:26.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:25 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/440667217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:31:26.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:25 smithi189 ceph-mon[34851]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:26.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:25 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/440667217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:31:26.090 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:31:26.466 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:27.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.049+0000 7fe47ae00640 1 -- 172.21.15.159:0/1945767458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe474102360 msgr2=0x7fe47410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:27.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.049+0000 7fe47ae00640 1 --2- 172.21.15.159:0/1945767458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe474102360 0x7fe47410e890 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009930 tx=0x7fe46c02a440 comp rx=0 tx=0).stop 2024-09-19T07:31:27.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.049+0000 7fe47ae00640 1 -- 172.21.15.159:0/1945767458 shutdown_connections 2024-09-19T07:31:27.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.049+0000 7fe47ae00640 1 --2- 172.21.15.159:0/1945767458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe474102360 0x7fe47410e890 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:27.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.049+0000 7fe47ae00640 1 --2- 172.21.15.159:0/1945767458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4741019c0 0x7fe474101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:27.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.049+0000 7fe47ae00640 1 --2- 172.21.15.159:0/1945767458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4741079c0 0x7fe474107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:27.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.049+0000 7fe47ae00640 1 -- 172.21.15.159:0/1945767458 >> 172.21.15.159:0/1945767458 conn(0x7fe4740fd740 msgr2=0x7fe4740ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:27.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.049+0000 7fe47ae00640 1 -- 172.21.15.159:0/1945767458 shutdown_connections 2024-09-19T07:31:27.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.049+0000 7fe47ae00640 1 -- 172.21.15.159:0/1945767458 wait complete. 2024-09-19T07:31:27.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.050+0000 7fe47ae00640 1 Processor -- start 2024-09-19T07:31:27.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.051+0000 7fe47ae00640 1 -- start start 2024-09-19T07:31:27.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.051+0000 7fe47ae00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4741019c0 0x7fe474198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:27.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.051+0000 7fe47ae00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe474102360 0x7fe474198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:27.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.051+0000 7fe47ae00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4741079c0 0x7fe47419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:27.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.051+0000 7fe47ae00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe474111e10 con 0x7fe474102360 2024-09-19T07:31:27.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.051+0000 7fe47ae00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe474111c90 con 0x7fe4741079c0 2024-09-19T07:31:27.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.051+0000 7fe47ae00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe474111f90 con 0x7fe4741019c0 2024-09-19T07:31:27.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.051+0000 7fe479a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4741019c0 0x7fe474198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:27.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.051+0000 7fe479000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe474102360 0x7fe474198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:27.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.051+0000 7fe479a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4741019c0 0x7fe474198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:45276/0 (socket says 172.21.15.159:45276) 2024-09-19T07:31:27.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.051+0000 7fe479a00640 1 -- 172.21.15.159:0/3617695952 learned_addr learned my addr 172.21.15.159:0/3617695952 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:27.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.051+0000 7fe47a400640 1 --2- 172.21.15.159:0/3617695952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4741079c0 0x7fe47419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:27.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.052+0000 7fe479a00640 1 -- 172.21.15.159:0/3617695952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4741079c0 msgr2=0x7fe47419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:27.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.052+0000 7fe479a00640 1 --2- 172.21.15.159:0/3617695952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4741079c0 0x7fe47419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:27.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.052+0000 7fe479a00640 1 -- 172.21.15.159:0/3617695952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe474102360 msgr2=0x7fe474198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:27.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.052+0000 7fe479a00640 1 --2- 172.21.15.159:0/3617695952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe474102360 0x7fe474198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:27.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.052+0000 7fe479a00640 1 -- 172.21.15.159:0/3617695952 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe47419d1e0 con 0x7fe4741019c0 2024-09-19T07:31:27.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.052+0000 7fe479000640 1 --2- 172.21.15.159:0/3617695952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe474102360 0x7fe474198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:31:27.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.052+0000 7fe479a00640 1 --2- 172.21.15.159:0/3617695952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4741019c0 0x7fe474198230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe46800eb90 tx=0x7fe46800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:27.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.052+0000 7fe472000640 1 -- 172.21.15.159:0/3617695952 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468004d20 con 0x7fe4741019c0 2024-09-19T07:31:27.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.052+0000 7fe47ae00640 1 -- 172.21.15.159:0/3617695952 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe47419d4d0 con 0x7fe4741019c0 2024-09-19T07:31:27.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.052+0000 7fe472000640 1 -- 172.21.15.159:0/3617695952 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe46800ee90 con 0x7fe4741019c0 2024-09-19T07:31:27.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.052+0000 7fe472000640 1 -- 172.21.15.159:0/3617695952 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468010830 con 0x7fe4741019c0 2024-09-19T07:31:27.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.053+0000 7fe47ae00640 1 -- 172.21.15.159:0/3617695952 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4741a4db0 con 0x7fe4741019c0 2024-09-19T07:31:27.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.054+0000 7fe47ae00640 1 -- 172.21.15.159:0/3617695952 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe474101e20 con 0x7fe4741019c0 2024-09-19T07:31:27.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.055+0000 7fe472000640 1 -- 172.21.15.159:0/3617695952 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fe468005020 con 0x7fe4741019c0 2024-09-19T07:31:27.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.057+0000 7fe472000640 1 --2- 172.21.15.159:0/3617695952 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe45c0a3240 0x7fe45c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:27.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.057+0000 7fe472000640 1 -- 172.21.15.159:0/3617695952 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7fe4680d0170 con 0x7fe4741019c0 2024-09-19T07:31:27.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.057+0000 7fe479000640 1 --2- 172.21.15.159:0/3617695952 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe45c0a3240 0x7fe45c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:27.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.057+0000 7fe479000640 1 --2- 172.21.15.159:0/3617695952 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe45c0a3240 0x7fe45c0a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe474106f50 tx=0x7fe46000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:27.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.061+0000 7fe472000640 1 -- 172.21.15.159:0/3617695952 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe46809cbf0 con 0x7fe4741019c0 2024-09-19T07:31:27.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.240+0000 7fe47ae00640 1 -- 172.21.15.159:0/3617695952 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe474199590 con 0x7fe4741019c0 2024-09-19T07:31:27.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.242+0000 7fe472000640 1 -- 172.21.15.159:0/3617695952 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24294 (secure 0 0 0) 0x7fe46809c4f0 con 0x7fe4741019c0 2024-09-19T07:31:27.244 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:31:27.244 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":96,"fsid":"c9d18048-7657-11ef-bceb-c7b262605968","created":"2024-09-19T07:22:32.059836+0000","modified":"2024-09-19T07:31:16.662906+0000","last_up_change":"2024-09-19T07:30:03.328697+0000","last_in_change":"2024-09-19T07:29:38.987579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:25:55.202457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:30:58.022056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:31:01.042290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:31:10.977289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"752f5466-fe04-408a-8566-86625947fd3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6803","nonce":2738900830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6805","nonce":2738900830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6809","nonce":2738900830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6807","nonce":2738900830}]},"public_addr":"172.21.15.159:6803/2738900830","cluster_addr":"172.21.15.159:6805/2738900830","heartbeat_back_addr":"172.21.15.159:6809/2738900830","heartbeat_front_addr":"172.21.15.159:6807/2738900830","state":["exists","up"]},{"osd":1,"uuid":"7513adda-ca82-4df4-aa99-3b050b410424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6801","nonce":265696178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6803","nonce":265696178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6807","nonce":265696178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6805","nonce":265696178}]},"public_addr":"172.21.15.189:6801/265696178","cluster_addr":"172.21.15.189:6803/265696178","heartbeat_back_addr":"172.21.15.189:6807/265696178","heartbeat_front_addr":"172.21.15.189:6805/265696178","state":["exists","up"]},{"osd":2,"uuid":"aa03be79-58d8-4eb7-a051-0b46c502b12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6801","nonce":1826565288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6803","nonce":1826565288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6807","nonce":1826565288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6805","nonce":1826565288}]},"public_addr":"172.21.15.203:6801/1826565288","cluster_addr":"172.21.15.203:6803/1826565288","heartbeat_back_addr":"172.21.15.203:6807/1826565288","heartbeat_front_addr":"172.21.15.203:6805/1826565288","state":["exists","up"]},{"osd":3,"uuid":"fcd734f1-f40f-4566-82ab-c4607039f23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6811","nonce":109457380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6813","nonce":109457380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6817","nonce":109457380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6815","nonce":109457380}]},"public_addr":"172.21.15.159:6811/109457380","cluster_addr":"172.21.15.159:6813/109457380","heartbeat_back_addr":"172.21.15.159:6817/109457380","heartbeat_front_addr":"172.21.15.159:6815/109457380","state":["exists","up"]},{"osd":4,"uuid":"bc344c9e-2267-4058-bbe1-9c270327db8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6809","nonce":1036888180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6811","nonce":1036888180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6815","nonce":1036888180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6813","nonce":1036888180}]},"public_addr":"172.21.15.189:6809/1036888180","cluster_addr":"172.21.15.189:6811/1036888180","heartbeat_back_addr":"172.21.15.189:6815/1036888180","heartbeat_front_addr":"172.21.15.189:6813/1036888180","state":["exists","up"]},{"osd":5,"uuid":"86c48303-929d-42f4-bf48-cbb32649f95f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6809","nonce":4212601512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6811","nonce":4212601512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6815","nonce":4212601512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6813","nonce":4212601512}]},"public_addr":"172.21.15.203:6809/4212601512","cluster_addr":"172.21.15.203:6811/4212601512","heartbeat_back_addr":"172.21.15.203:6815/4212601512","heartbeat_front_addr":"172.21.15.203:6813/4212601512","state":["exists","up"]},{"osd":6,"uuid":"f0176ed1-c3a7-4d75-b213-fe88fab4441a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6819","nonce":3817684578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6821","nonce":3817684578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6825","nonce":3817684578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6823","nonce":3817684578}]},"public_addr":"172.21.15.159:6819/3817684578","cluster_addr":"172.21.15.159:6821/3817684578","heartbeat_back_addr":"172.21.15.159:6825/3817684578","heartbeat_front_addr":"172.21.15.159:6823/3817684578","state":["exists","up"]},{"osd":7,"uuid":"921bd59d-9415-4559-8e9f-035d336afd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6817","nonce":3936826456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6819","nonce":3936826456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6823","nonce":3936826456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6821","nonce":3936826456}]},"public_addr":"172.21.15.189:6817/3936826456","cluster_addr":"172.21.15.189:6819/3936826456","heartbeat_back_addr":"172.21.15.189:6823/3936826456","heartbeat_front_addr":"172.21.15.189:6821/3936826456","state":["exists","up"]},{"osd":8,"uuid":"ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6817","nonce":4062926645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6819","nonce":4062926645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6823","nonce":4062926645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6821","nonce":4062926645}]},"public_addr":"172.21.15.203:6817/4062926645","cluster_addr":"172.21.15.203:6819/4062926645","heartbeat_back_addr":"172.21.15.203:6823/4062926645","heartbeat_front_addr":"172.21.15.203:6821/4062926645","state":["exists","up"]},{"osd":9,"uuid":"df29a10f-1c60-41cb-9c2d-bc9f1cb174bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6827","nonce":4185848747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6829","nonce":4185848747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6833","nonce":4185848747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6831","nonce":4185848747}]},"public_addr":"172.21.15.159:6827/4185848747","cluster_addr":"172.21.15.159:6829/4185848747","heartbeat_back_addr":"172.21.15.159:6833/4185848747","heartbeat_front_addr":"172.21.15.159:6831/4185848747","state":["exists","up"]},{"osd":10,"uuid":"ccc440b4-4682-4998-bb53-95e24423cd1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6825","nonce":3018377605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6827","nonce":3018377605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6831","nonce":3018377605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6829","nonce":3018377605}]},"public_addr":"172.21.15.189:6825/3018377605","cluster_addr":"172.21.15.189:6827/3018377605","heartbeat_back_addr":"172.21.15.189:6831/3018377605","heartbeat_front_addr":"172.21.15.189:6829/3018377605","state":["exists","up"]},{"osd":11,"uuid":"e84afc5f-8051-4341-8fd0-3afb482ab4df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6825","nonce":2807503891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6827","nonce":2807503891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6831","nonce":2807503891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6829","nonce":2807503891}]},"public_addr":"172.21.15.203:6825/2807503891","cluster_addr":"172.21.15.203:6827/2807503891","heartbeat_back_addr":"172.21.15.203:6831/2807503891","heartbeat_front_addr":"172.21.15.203:6829/2807503891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:00.093677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:25.477205+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:51.345020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:21.077603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:48.822640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:13.516029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:44.382228+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:10.810700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:36.817155+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:07.728553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:34.747540+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:01.249332+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:0/2383222735":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1561371050":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6800/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:6800/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/2108730247":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/3393992615":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1366863724":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/1628344860":"2024-09-20T07:23:03.089190+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:31:27.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.245+0000 7fe47ae00640 1 -- 172.21.15.159:0/3617695952 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe45c0a3240 msgr2=0x7fe45c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:27.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.245+0000 7fe47ae00640 1 --2- 172.21.15.159:0/3617695952 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe45c0a3240 0x7fe45c0a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe474106f50 tx=0x7fe46000b410 comp rx=0 tx=0).stop 2024-09-19T07:31:27.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.245+0000 7fe47ae00640 1 -- 172.21.15.159:0/3617695952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4741019c0 msgr2=0x7fe474198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:27.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.245+0000 7fe47ae00640 1 --2- 172.21.15.159:0/3617695952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4741019c0 0x7fe474198230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe46800eb90 tx=0x7fe46800c6a0 comp rx=0 tx=0).stop 2024-09-19T07:31:27.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.245+0000 7fe47ae00640 1 -- 172.21.15.159:0/3617695952 shutdown_connections 2024-09-19T07:31:27.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.245+0000 7fe47ae00640 1 --2- 172.21.15.159:0/3617695952 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe45c0a3240 0x7fe45c0a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:27.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.245+0000 7fe47ae00640 1 --2- 172.21.15.159:0/3617695952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4741079c0 0x7fe47419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:27.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.245+0000 7fe47ae00640 1 --2- 172.21.15.159:0/3617695952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe474102360 0x7fe474198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:27.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.245+0000 7fe47ae00640 1 --2- 172.21.15.159:0/3617695952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4741019c0 0x7fe474198230 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:27.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.245+0000 7fe47ae00640 1 -- 172.21.15.159:0/3617695952 >> 172.21.15.159:0/3617695952 conn(0x7fe4740fd740 msgr2=0x7fe4740fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:27.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.245+0000 7fe47ae00640 1 -- 172.21.15.159:0/3617695952 shutdown_connections 2024-09-19T07:31:27.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:27.245+0000 7fe47ae00640 1 -- 172.21.15.159:0/3617695952 wait complete. 2024-09-19T07:31:28.003 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:27 smithi159 ceph-mon[30472]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:28.004 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:27 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3617695952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:31:28.041 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T07:31:03.518840+0000', 'modified': '2024-09-19T07:31:16.663062+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24542}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24542': {'gid': 24542, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.189:6839/2361818663', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 2361818663}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 2361818663}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24542, 'qdb_cluster': [24542]} 2024-09-19T07:31:28.041 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T07:31:28.041 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T07:31:28.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:27 smithi189 ceph-mon[34851]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:28.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:27 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3617695952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:31:28.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:27 smithi203 ceph-mon[35335]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:28.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:27 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3617695952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:31:28.413 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:28.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.959+0000 7f85e9000640 1 -- 172.21.15.159:0/1385007625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85e40ffc40 msgr2=0x7f85e41000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:28.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.959+0000 7f85e9000640 1 --2- 172.21.15.159:0/1385007625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85e40ffc40 0x7f85e41000c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009930 tx=0x7f85cc02a420 comp rx=0 tx=0).stop 2024-09-19T07:31:28.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.959+0000 7f85e9000640 1 -- 172.21.15.159:0/1385007625 shutdown_connections 2024-09-19T07:31:28.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.959+0000 7f85e9000640 1 --2- 172.21.15.159:0/1385007625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85e4100600 0x7f85e410e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.959+0000 7f85e9000640 1 --2- 172.21.15.159:0/1385007625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85e40ffc40 0x7f85e41000c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.959+0000 7f85e9000640 1 --2- 172.21.15.159:0/1385007625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85e4106c80 0x7f85e4107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.959+0000 7f85e9000640 1 -- 172.21.15.159:0/1385007625 >> 172.21.15.159:0/1385007625 conn(0x7f85e40fba90 msgr2=0x7f85e40fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:28.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.959+0000 7f85e9000640 1 -- 172.21.15.159:0/1385007625 shutdown_connections 2024-09-19T07:31:28.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.959+0000 7f85e9000640 1 -- 172.21.15.159:0/1385007625 wait complete. 2024-09-19T07:31:28.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.960+0000 7f85e9000640 1 Processor -- start 2024-09-19T07:31:28.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.960+0000 7f85e9000640 1 -- start start 2024-09-19T07:31:28.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.960+0000 7f85e9000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85e40ffc40 0x7f85e419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:28.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.960+0000 7f85e9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85e4100600 0x7f85e419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:28.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.960+0000 7f85e9000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85e4106c80 0x7f85e419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:28.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.961+0000 7f85e9000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85e4111f70 con 0x7f85e4100600 2024-09-19T07:31:28.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.961+0000 7f85e9000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f85e4111df0 con 0x7f85e4106c80 2024-09-19T07:31:28.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.961+0000 7f85e9000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f85e41120f0 con 0x7f85e40ffc40 2024-09-19T07:31:28.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.961+0000 7f85dbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85e4100600 0x7f85e419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:28.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.961+0000 7f85e1e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85e40ffc40 0x7f85e419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:28.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.961+0000 7f85dbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85e4100600 0x7f85e419ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59874/0 (socket says 172.21.15.159:59874) 2024-09-19T07:31:28.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.961+0000 7f85e1e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85e40ffc40 0x7f85e419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:45308/0 (socket says 172.21.15.159:45308) 2024-09-19T07:31:28.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.961+0000 7f85dbe00640 1 -- 172.21.15.159:0/2570690024 learned_addr learned my addr 172.21.15.159:0/2570690024 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:28.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.961+0000 7f85e2800640 1 --2- 172.21.15.159:0/2570690024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85e4106c80 0x7f85e419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:28.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.961+0000 7f85dbe00640 1 -- 172.21.15.159:0/2570690024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85e40ffc40 msgr2=0x7f85e419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:28.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.961+0000 7f85dbe00640 1 --2- 172.21.15.159:0/2570690024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85e40ffc40 0x7f85e419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.961+0000 7f85dbe00640 1 -- 172.21.15.159:0/2570690024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85e4106c80 msgr2=0x7f85e419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:28.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.961+0000 7f85dbe00640 1 --2- 172.21.15.159:0/2570690024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85e4106c80 0x7f85e419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.961+0000 7f85dbe00640 1 -- 172.21.15.159:0/2570690024 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e419f5a0 con 0x7f85e4100600 2024-09-19T07:31:28.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.962+0000 7f85dbe00640 1 --2- 172.21.15.159:0/2570690024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85e4100600 0x7f85e419ab30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f85cc004770 tx=0x7f85cc003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:28.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.962+0000 7f85daa00640 1 -- 172.21.15.159:0/2570690024 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc03f070 con 0x7f85e4100600 2024-09-19T07:31:28.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.962+0000 7f85daa00640 1 -- 172.21.15.159:0/2570690024 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85cc031c20 con 0x7f85e4100600 2024-09-19T07:31:28.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.962+0000 7f85daa00640 1 -- 172.21.15.159:0/2570690024 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc035030 con 0x7f85e4100600 2024-09-19T07:31:28.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.962+0000 7f85e9000640 1 -- 172.21.15.159:0/2570690024 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e419f830 con 0x7f85e4100600 2024-09-19T07:31:28.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.962+0000 7f85e9000640 1 -- 172.21.15.159:0/2570690024 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e41a7170 con 0x7f85e4100600 2024-09-19T07:31:28.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.965+0000 7f85daa00640 1 -- 172.21.15.159:0/2570690024 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f85cc042070 con 0x7f85e4100600 2024-09-19T07:31:28.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.965+0000 7f85ebc01740 1 Processor -- start 2024-09-19T07:31:28.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.965+0000 7f85ebc01740 1 -- start start 2024-09-19T07:31:28.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.966+0000 7f85ebc01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc28b40 0x55c80dc28f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:28.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.966+0000 7f85ebc01740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc29460 0x55c80dc337e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:28.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.966+0000 7f85ebc01740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc33d20 0x55c80dc36110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:28.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.966+0000 7f85ebc01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c80db860b0 con 0x55c80dc28b40 2024-09-19T07:31:28.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.966+0000 7f85ebc01740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c80db2dab0 con 0x55c80dc33d20 2024-09-19T07:31:28.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.966+0000 7f85ebc01740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c80db82070 con 0x55c80dc29460 2024-09-19T07:31:28.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.966+0000 7f85e2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc28b40 0x55c80dc28f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:28.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.966+0000 7f85e2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc28b40 0x55c80dc28f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59888/0 (socket says 172.21.15.159:59888) 2024-09-19T07:31:28.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.966+0000 7f85e2800640 1 -- 172.21.15.159:0/801231709 learned_addr learned my addr 172.21.15.159:0/801231709 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:28.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.966+0000 7f85dbe00640 1 --2- 172.21.15.159:0/801231709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc33d20 0x55c80dc36110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:28.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.966+0000 7f85e1e00640 1 --2- 172.21.15.159:0/801231709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc29460 0x55c80dc337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:28.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.966+0000 7f85daa00640 1 --2- 172.21.15.159:0/2570690024 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f85c80a32c0 0x7f85c80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:28.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.967+0000 7f85dbe00640 1 -- 172.21.15.159:0/801231709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc29460 msgr2=0x55c80dc337e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:31:28.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.967+0000 7f85dbe00640 1 --2- 172.21.15.159:0/801231709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc29460 0x55c80dc337e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.967+0000 7f85dbe00640 1 -- 172.21.15.159:0/801231709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc28b40 msgr2=0x55c80dc28f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:31:28.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.967+0000 7f85dbe00640 1 --2- 172.21.15.159:0/801231709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc28b40 0x55c80dc28f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.967+0000 7f85dbe00640 1 -- 172.21.15.159:0/801231709 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c80db3a4a0 con 0x55c80dc33d20 2024-09-19T07:31:28.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.967+0000 7f85e2800640 1 --2- 172.21.15.159:0/2570690024 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f85c80a32c0 0x7f85c80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:28.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.967+0000 7f85daa00640 1 -- 172.21.15.159:0/2570690024 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f85cc0f04a0 con 0x7f85e4100600 2024-09-19T07:31:28.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.967+0000 7f85dbe00640 1 --2- 172.21.15.159:0/801231709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc33d20 0x55c80dc36110 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f85cc0edcd0 tx=0x7f85cc087310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=940091e312d3b7f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:28.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.967+0000 7f85d3400640 1 -- 172.21.15.159:0/801231709 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc08e710 con 0x55c80dc33d20 2024-09-19T07:31:28.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.967+0000 7f85d3400640 1 -- 172.21.15.159:0/801231709 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85cc0a70c0 con 0x55c80dc33d20 2024-09-19T07:31:28.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.967+0000 7f85d3400640 1 -- 172.21.15.159:0/801231709 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc08c5b0 con 0x55c80dc33d20 2024-09-19T07:31:28.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.968+0000 7f85e2800640 1 --2- 172.21.15.159:0/2570690024 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f85c80a32c0 0x7f85c80a5780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009fd0 tx=0x7f85dc004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:28.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.968+0000 7f85ebc01740 1 -- 172.21.15.159:0/801231709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc33d20 msgr2=0x55c80dc36110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:28.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.968+0000 7f85ebc01740 1 --2- 172.21.15.159:0/801231709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc33d20 0x55c80dc36110 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f85cc0edcd0 tx=0x7f85cc087310 comp rx=0 tx=0).stop 2024-09-19T07:31:28.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.968+0000 7f85ebc01740 1 -- 172.21.15.159:0/801231709 shutdown_connections 2024-09-19T07:31:28.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.968+0000 7f85ebc01740 1 --2- 172.21.15.159:0/801231709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc33d20 0x55c80dc36110 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.968+0000 7f85ebc01740 1 --2- 172.21.15.159:0/801231709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc29460 0x55c80dc337e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.968+0000 7f85ebc01740 1 --2- 172.21.15.159:0/801231709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc28b40 0x55c80dc28f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.968+0000 7f85ebc01740 1 -- 172.21.15.159:0/801231709 >> 172.21.15.159:0/801231709 conn(0x55c80dc250d0 msgr2=0x55c80dc254b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:28.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.968+0000 7f85ebc01740 1 -- 172.21.15.159:0/801231709 shutdown_connections 2024-09-19T07:31:28.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.968+0000 7f85ebc01740 1 -- 172.21.15.159:0/801231709 wait complete. 2024-09-19T07:31:28.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.969+0000 7f85ebc01740 1 -- start start 2024-09-19T07:31:28.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.969+0000 7f85ebc01740 1 Processor -- start 2024-09-19T07:31:28.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.969+0000 7f85ebc01740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc28b40 0x55c80dc30b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:28.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.969+0000 7f85ebc01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc29460 0x55c80dc310c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:28.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.969+0000 7f85ebc01740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc33d20 0x55c80dc31600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:28.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.969+0000 7f85ebc01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c80dc366b0 con 0x55c80dc29460 2024-09-19T07:31:28.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.969+0000 7f85ebc01740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c80db0e920 con 0x55c80dc28b40 2024-09-19T07:31:28.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.969+0000 7f85ebc01740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c80dc36830 con 0x55c80dc33d20 2024-09-19T07:31:28.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.969+0000 7f85ebc01740 10 client.0 _ll_register_callbacks cb 0x55c80db328a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:31:28.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.970+0000 7f85e1e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc28b40 0x55c80dc30b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:28.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.970+0000 7f85e1e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc28b40 0x55c80dc30b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:46270/0 (socket says 172.21.15.159:46270) 2024-09-19T07:31:28.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.970+0000 7f85e1e00640 1 -- 172.21.15.159:0/140502579 learned_addr learned my addr 172.21.15.159:0/140502579 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:28.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.970+0000 7f85dbe00640 1 --2- 172.21.15.159:0/140502579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc29460 0x55c80dc310c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:28.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.970+0000 7f85e2800640 1 --2- 172.21.15.159:0/140502579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc33d20 0x55c80dc31600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:28.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.970+0000 7f85e1e00640 1 -- 172.21.15.159:0/140502579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc33d20 msgr2=0x55c80dc31600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:31:28.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.970+0000 7f85e1e00640 1 --2- 172.21.15.159:0/140502579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc33d20 0x55c80dc31600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.970+0000 7f85e1e00640 1 -- 172.21.15.159:0/140502579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc29460 msgr2=0x55c80dc310c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:31:28.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.970+0000 7f85e1e00640 1 --2- 172.21.15.159:0/140502579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc29460 0x55c80dc310c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.970+0000 7f85e1e00640 1 -- 172.21.15.159:0/140502579 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c80d8f72b0 con 0x55c80dc28b40 2024-09-19T07:31:28.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.970+0000 7f85e1e00640 1 --2- 172.21.15.159:0/140502579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc28b40 0x55c80dc30b80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f85d4012770 tx=0x7f85d4012c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b653d57f9a7ebd91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:28.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.970+0000 7f85c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-19T07:31:28.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.971+0000 7f85c7e00640 1 -- 172.21.15.159:0/140502579 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d400f920 con 0x55c80dc28b40 2024-09-19T07:31:28.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.971+0000 7f85ebc01740 1 -- 172.21.15.159:0/140502579 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c80db93620 con 0x55c80dc28b40 2024-09-19T07:31:28.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.971+0000 7f85c7e00640 1 -- 172.21.15.159:0/140502579 <== mon.2 v2:172.21.15.189:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f85d400fac0 con 0x55c80dc28b40 2024-09-19T07:31:28.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.971+0000 7f85c7e00640 1 -- 172.21.15.159:0/140502579 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f85d400fcc0 con 0x55c80dc28b40 2024-09-19T07:31:28.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.971+0000 7f85ebc01740 10 client.24638 fetch_fsmap learned FSMap version 17 2024-09-19T07:31:28.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.971+0000 7f85ebc01740 1 -- 172.21.15.159:0/140502579 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55c80dc42330 con 0x55c80dc28b40 2024-09-19T07:31:28.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.971+0000 7f85c7e00640 1 -- 172.21.15.159:0/140502579 <== mon.2 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c80db93620 con 0x55c80dc28b40 2024-09-19T07:31:28.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.971+0000 7f85c7e00640 1 -- 172.21.15.159:0/140502579 <== mon.2 v2:172.21.15.189:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f85d401b3c0 con 0x55c80dc28b40 2024-09-19T07:31:28.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.971+0000 7f85ebc01740 10 client.24638 fetch_fsmap finished waiting for FSMap version 17 2024-09-19T07:31:28.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.972+0000 7f85ebc01740 10 client.24638 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T07:31:28.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.972+0000 7f85ebc01740 20 client.24638 populate_metadata read hostname 'smithi159' 2024-09-19T07:31:28.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.972+0000 7f85ebc01740 1 --2- 172.21.15.159:0/140502579 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x55c80dc44960 0x55c80dc45dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:28.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.972+0000 7f85ebc01740 4 client.24638 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:31:28.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.972+0000 7f85ebc01740 1 -- 172.21.15.159:0/140502579 --> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c80db8db30 con 0x55c80dc44960 2024-09-19T07:31:28.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.972+0000 7f85dbe00640 1 --2- 172.21.15.159:0/140502579 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x55c80dc44960 0x55c80dc45dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:28.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.972+0000 7f85dbe00640 1 --2- 172.21.15.159:0/140502579 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x55c80dc44960 0x55c80dc45dd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e7f51fcf7c28b11 server_cookie=c20a412d56543588 in_seq=0 out_seq=0 2024-09-19T07:31:28.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.972+0000 7f85c7e00640 10 client.24638 ms_handle_connect on v2:172.21.15.189:6838/2361818663 2024-09-19T07:31:28.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.973+0000 7f85c7e00640 1 -- 172.21.15.159:0/140502579 <== mds.0 v2:172.21.15.189:6838/2361818663 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c80db8db30 con 0x55c80dc44960 2024-09-19T07:31:28.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.973+0000 7f85c7e00640 10 client.24638 handle_command_reply: tid=0 2024-09-19T07:31:28.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.973+0000 7f85ebc01740 1 client.24638 shutdown 2024-09-19T07:31:28.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.974+0000 7f85ebc01740 1 -- 172.21.15.159:0/140502579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc28b40 msgr2=0x55c80dc30b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:28.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.974+0000 7f85ebc01740 1 --2- 172.21.15.159:0/140502579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc28b40 0x55c80dc30b80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f85d4012770 tx=0x7f85d4012c30 comp rx=0 tx=0).stop 2024-09-19T07:31:28.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.974+0000 7f85ebc01740 1 -- 172.21.15.159:0/140502579 shutdown_connections 2024-09-19T07:31:28.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.974+0000 7f85ebc01740 1 --2- 172.21.15.159:0/140502579 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x55c80dc44960 0x55c80dc45dd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.974+0000 7f85ebc01740 1 --2- 172.21.15.159:0/140502579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc33d20 0x55c80dc31600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.974+0000 7f85ebc01740 1 --2- 172.21.15.159:0/140502579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc29460 0x55c80dc310c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.974+0000 7f85ebc01740 1 --2- 172.21.15.159:0/140502579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc28b40 0x55c80dc30b80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.974+0000 7f85c7e00640 0 client.24638 ms_handle_reset on v2:172.21.15.189:6838/2361818663 2024-09-19T07:31:28.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.974+0000 7f85ebc01740 1 -- 172.21.15.159:0/140502579 >> 172.21.15.159:0/140502579 conn(0x55c80dc250d0 msgr2=0x55c80dc2ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:28.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.974+0000 7f85ebc01740 1 -- 172.21.15.159:0/140502579 shutdown_connections 2024-09-19T07:31:28.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.974+0000 7f85ebc01740 1 -- 172.21.15.159:0/140502579 wait complete. 2024-09-19T07:31:28.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.974+0000 7f85ebc01740 20 client.24638 trim_cache size 0 max 16384 2024-09-19T07:31:28.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.992+0000 7f85ebc01740 1 Processor -- start 2024-09-19T07:31:28.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.992+0000 7f85ebc01740 1 -- start start 2024-09-19T07:31:28.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.992+0000 7f85ebc01740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc28b40 0x55c80dc2b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:28.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.992+0000 7f85ebc01740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc29460 0x55c80dc310f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:28.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.992+0000 7f85ebc01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc33d20 0x55c80dc31630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:28.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.992+0000 7f85ebc01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c80dc44110 con 0x55c80dc33d20 2024-09-19T07:31:28.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.992+0000 7f85ebc01740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c80dc447e0 con 0x55c80dc29460 2024-09-19T07:31:28.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.992+0000 7f85ebc01740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c80dc44eb0 con 0x55c80dc28b40 2024-09-19T07:31:28.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.993+0000 7f85e2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc33d20 0x55c80dc31630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:28.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.993+0000 7f85e2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc33d20 0x55c80dc31630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59914/0 (socket says 172.21.15.159:59914) 2024-09-19T07:31:28.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.993+0000 7f85e2800640 1 -- 172.21.15.159:0/4119893459 learned_addr learned my addr 172.21.15.159:0/4119893459 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:28.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.993+0000 7f85e1e00640 1 --2- 172.21.15.159:0/4119893459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc28b40 0x55c80dc2b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:28.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.993+0000 7f85dbe00640 1 --2- 172.21.15.159:0/4119893459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc29460 0x55c80dc310f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:28.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.993+0000 7f85e1e00640 1 -- 172.21.15.159:0/4119893459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc29460 msgr2=0x55c80dc310f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:31:28.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.993+0000 7f85e1e00640 1 --2- 172.21.15.159:0/4119893459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc29460 0x55c80dc310f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.993+0000 7f85e1e00640 1 -- 172.21.15.159:0/4119893459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc33d20 msgr2=0x55c80dc31630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:31:28.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.993+0000 7f85e1e00640 1 --2- 172.21.15.159:0/4119893459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc33d20 0x55c80dc31630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.993+0000 7f85e1e00640 1 -- 172.21.15.159:0/4119893459 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c80dc24f00 con 0x55c80dc28b40 2024-09-19T07:31:28.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.993+0000 7f85e2800640 1 --2- 172.21.15.159:0/4119893459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc33d20 0x55c80dc31630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:31:28.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.993+0000 7f85e1e00640 1 --2- 172.21.15.159:0/4119893459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc28b40 0x55c80dc2b5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f85d401b880 tx=0x7f85d4012420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e051e3fbbf59516e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:28.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.994+0000 7f85d2a00640 1 -- 172.21.15.159:0/4119893459 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d400f920 con 0x55c80dc28b40 2024-09-19T07:31:28.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.994+0000 7f85d2a00640 1 -- 172.21.15.159:0/4119893459 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85d400fac0 con 0x55c80dc28b40 2024-09-19T07:31:28.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.994+0000 7f85d2a00640 1 -- 172.21.15.159:0/4119893459 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d40035f0 con 0x55c80dc28b40 2024-09-19T07:31:28.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.994+0000 7f85ebc01740 1 -- 172.21.15.159:0/4119893459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc28b40 msgr2=0x55c80dc2b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:28.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.994+0000 7f85ebc01740 1 --2- 172.21.15.159:0/4119893459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc28b40 0x55c80dc2b5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f85d401b880 tx=0x7f85d4012420 comp rx=0 tx=0).stop 2024-09-19T07:31:28.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.994+0000 7f85ebc01740 1 -- 172.21.15.159:0/4119893459 shutdown_connections 2024-09-19T07:31:28.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.994+0000 7f85ebc01740 1 --2- 172.21.15.159:0/4119893459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc33d20 0x55c80dc31630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.994+0000 7f85ebc01740 1 --2- 172.21.15.159:0/4119893459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc29460 0x55c80dc310f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.994+0000 7f85ebc01740 1 --2- 172.21.15.159:0/4119893459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc28b40 0x55c80dc2b5a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:28.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.994+0000 7f85ebc01740 1 -- 172.21.15.159:0/4119893459 >> 172.21.15.159:0/4119893459 conn(0x55c80dc250d0 msgr2=0x55c80dc2bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:28.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.995+0000 7f85ebc01740 1 -- 172.21.15.159:0/4119893459 shutdown_connections 2024-09-19T07:31:28.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.995+0000 7f85ebc01740 1 -- 172.21.15.159:0/4119893459 wait complete. 2024-09-19T07:31:28.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.995+0000 7f85ebc01740 1 -- start start 2024-09-19T07:31:28.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.995+0000 7f85ebc01740 1 Processor -- start 2024-09-19T07:31:28.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.996+0000 7f85ebc01740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc28b40 0x55c80dc4e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:28.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.996+0000 7f85ebc01740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc29460 0x55c80dc4e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:28.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.996+0000 7f85ebc01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc33d20 0x55c80dc48fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:29.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.996+0000 7f85ebc01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c80dc45c50 con 0x55c80dc33d20 2024-09-19T07:31:29.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.996+0000 7f85ebc01740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c80dc45580 con 0x55c80dc29460 2024-09-19T07:31:29.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.996+0000 7f85ebc01740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c80dc46320 con 0x55c80dc28b40 2024-09-19T07:31:29.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.996+0000 7f85ebc01740 10 client.0 _ll_register_callbacks cb 0x7f85d4009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:31:29.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.996+0000 7f85dbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc29460 0x55c80dc4e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:29.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.996+0000 7f85dbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc29460 0x55c80dc4e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:46286/0 (socket says 172.21.15.159:46286) 2024-09-19T07:31:29.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.996+0000 7f85dbe00640 1 -- 172.21.15.159:0/2545297215 learned_addr learned my addr 172.21.15.159:0/2545297215 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:29.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.996+0000 7f85e2800640 1 --2- 172.21.15.159:0/2545297215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc33d20 0x55c80dc48fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:29.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.996+0000 7f85e1e00640 1 --2- 172.21.15.159:0/2545297215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc28b40 0x55c80dc4e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:29.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.996+0000 7f85dbe00640 1 -- 172.21.15.159:0/2545297215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc28b40 msgr2=0x55c80dc4e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:31:29.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.996+0000 7f85dbe00640 1 --2- 172.21.15.159:0/2545297215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc28b40 0x55c80dc4e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:29.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.996+0000 7f85dbe00640 1 -- 172.21.15.159:0/2545297215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc33d20 msgr2=0x55c80dc48fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:31:29.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.996+0000 7f85dbe00640 1 --2- 172.21.15.159:0/2545297215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc33d20 0x55c80dc48fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:29.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.996+0000 7f85dbe00640 1 -- 172.21.15.159:0/2545297215 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c80dc49520 con 0x55c80dc29460 2024-09-19T07:31:29.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.997+0000 7f85dbe00640 1 --2- 172.21.15.159:0/2545297215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc29460 0x55c80dc4e570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f85cc005d00 tx=0x7f85cc0bc510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9142521fed347dcb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:29.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.997+0000 7f85c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-19T07:31:29.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.997+0000 7f85c7e00640 1 -- 172.21.15.159:0/2545297215 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc049040 con 0x55c80dc29460 2024-09-19T07:31:29.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.997+0000 7f85ebc01740 1 -- 172.21.15.159:0/2545297215 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d400fac0 con 0x55c80dc29460 2024-09-19T07:31:29.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.997+0000 7f85c7e00640 1 -- 172.21.15.159:0/2545297215 <== mon.2 v2:172.21.15.189:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f85cc0b8760 con 0x55c80dc29460 2024-09-19T07:31:29.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.997+0000 7f85c7e00640 1 -- 172.21.15.159:0/2545297215 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f85cc0f02a0 con 0x55c80dc29460 2024-09-19T07:31:29.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.997+0000 7f85ebc01740 10 client.24644 fetch_fsmap learned FSMap version 17 2024-09-19T07:31:29.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.997+0000 7f85ebc01740 1 -- 172.21.15.159:0/2545297215 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55c80dcaf7f0 con 0x55c80dc29460 2024-09-19T07:31:29.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.997+0000 7f85c7e00640 1 -- 172.21.15.159:0/2545297215 <== mon.2 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85d400fac0 con 0x55c80dc29460 2024-09-19T07:31:29.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.998+0000 7f85c7e00640 1 -- 172.21.15.159:0/2545297215 <== mon.2 v2:172.21.15.189:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f85cc0edd80 con 0x55c80dc29460 2024-09-19T07:31:29.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.998+0000 7f85ebc01740 10 client.24644 fetch_fsmap finished waiting for FSMap version 17 2024-09-19T07:31:29.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.998+0000 7f85ebc01740 10 client.24644 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T07:31:29.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.998+0000 7f85ebc01740 20 client.24644 populate_metadata read hostname 'smithi159' 2024-09-19T07:31:29.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.998+0000 7f85ebc01740 1 --2- 172.21.15.159:0/2545297215 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x55c80dcb4180 0x55c80dcb2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:29.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.998+0000 7f85ebc01740 4 client.24644 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:31:29.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.998+0000 7f85ebc01740 1 -- 172.21.15.159:0/2545297215 --> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] -- command(tid 0: {"prefix": "status"}) -- 0x55c80db09f80 con 0x55c80dcb4180 2024-09-19T07:31:29.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.998+0000 7f85e1e00640 1 --2- 172.21.15.159:0/2545297215 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x55c80dcb4180 0x55c80dcb2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:29.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.998+0000 7f85e1e00640 1 --2- 172.21.15.159:0/2545297215 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x55c80dcb4180 0x55c80dcb2e90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6146e7e6672b6682 server_cookie=47c2aa21db55a263 in_seq=0 out_seq=0 2024-09-19T07:31:29.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.999+0000 7f85c7e00640 10 client.24644 ms_handle_connect on v2:172.21.15.189:6838/2361818663 2024-09-19T07:31:29.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.999+0000 7f85c7e00640 1 -- 172.21.15.159:0/2545297215 <== mds.0 v2:172.21.15.189:6838/2361818663 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c80db09f80 con 0x55c80dcb4180 2024-09-19T07:31:29.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.999+0000 7f85c7e00640 10 client.24644 handle_command_reply: tid=0 2024-09-19T07:31:29.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.999+0000 7f85ebc01740 1 client.24644 shutdown 2024-09-19T07:31:29.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.999+0000 7f85ebc01740 1 -- 172.21.15.159:0/2545297215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc29460 msgr2=0x55c80dc4e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:29.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.999+0000 7f85ebc01740 1 --2- 172.21.15.159:0/2545297215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc29460 0x55c80dc4e570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f85cc005d00 tx=0x7f85cc0bc510 comp rx=0 tx=0).stop 2024-09-19T07:31:29.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.999+0000 7f85ebc01740 1 -- 172.21.15.159:0/2545297215 shutdown_connections 2024-09-19T07:31:29.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.999+0000 7f85ebc01740 1 --2- 172.21.15.159:0/2545297215 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x55c80dcb4180 0x55c80dcb2e90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:29.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.999+0000 7f85ebc01740 1 --2- 172.21.15.159:0/2545297215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c80dc33d20 0x55c80dc48fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:29.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.999+0000 7f85c7e00640 0 client.24644 ms_handle_reset on v2:172.21.15.189:6838/2361818663 2024-09-19T07:31:29.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.999+0000 7f85ebc01740 1 --2- 172.21.15.159:0/2545297215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c80dc29460 0x55c80dc4e570 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:29.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.999+0000 7f85ebc01740 1 --2- 172.21.15.159:0/2545297215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c80dc28b40 0x55c80dc4e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:29.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.999+0000 7f85ebc01740 1 -- 172.21.15.159:0/2545297215 >> 172.21.15.159:0/2545297215 conn(0x55c80dc250d0 msgr2=0x55c80dc1b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:29.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.999+0000 7f85ebc01740 1 -- 172.21.15.159:0/2545297215 shutdown_connections 2024-09-19T07:31:29.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.999+0000 7f85ebc01740 1 -- 172.21.15.159:0/2545297215 wait complete. 2024-09-19T07:31:29.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:28.999+0000 7f85ebc01740 20 client.24644 trim_cache size 0 max 16384 2024-09-19T07:31:29.005 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-19T07:31:29.005 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "c9d18048-7657-11ef-bceb-c7b262605968", 2024-09-19T07:31:29.005 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 0, 2024-09-19T07:31:29.005 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 24542, 2024-09-19T07:31:29.005 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-09-19T07:31:29.005 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-09-19T07:31:29.005 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-09-19T07:31:29.005 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 24.571524663999998, 2024-09-19T07:31:29.005 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 17, 2024-09-19T07:31:29.005 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 96, 2024-09-19T07:31:29.006 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 95, 2024-09-19T07:31:29.006 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 39.366401168000003 2024-09-19T07:31:29.006 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-19T07:31:29.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:29.000+0000 7f85e9000640 1 -- 172.21.15.159:0/2570690024 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f85c80a32c0 msgr2=0x7f85c80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:29.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:29.000+0000 7f85e9000640 1 --2- 172.21.15.159:0/2570690024 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f85c80a32c0 0x7f85c80a5780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009fd0 tx=0x7f85dc004a40 comp rx=0 tx=0).stop 2024-09-19T07:31:29.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:29.000+0000 7f85e9000640 1 -- 172.21.15.159:0/2570690024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85e4100600 msgr2=0x7f85e419ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:29.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:29.000+0000 7f85e9000640 1 --2- 172.21.15.159:0/2570690024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85e4100600 0x7f85e419ab30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f85cc004770 tx=0x7f85cc003b90 comp rx=0 tx=0).stop 2024-09-19T07:31:29.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:29.000+0000 7f85e9000640 1 -- 172.21.15.159:0/2570690024 shutdown_connections 2024-09-19T07:31:29.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:29.000+0000 7f85e9000640 1 --2- 172.21.15.159:0/2570690024 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f85c80a32c0 0x7f85c80a5780 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:29.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:29.000+0000 7f85e9000640 1 --2- 172.21.15.159:0/2570690024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85e4106c80 0x7f85e419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:29.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:29.000+0000 7f85e9000640 1 --2- 172.21.15.159:0/2570690024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85e4100600 0x7f85e419ab30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:29.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:29.000+0000 7f85e9000640 1 --2- 172.21.15.159:0/2570690024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85e40ffc40 0x7f85e419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:29.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:29.000+0000 7f85e9000640 1 -- 172.21.15.159:0/2570690024 >> 172.21.15.159:0/2570690024 conn(0x7f85e40fba90 msgr2=0x7f85e40fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:29.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:29.000+0000 7f85e9000640 1 -- 172.21.15.159:0/2570690024 shutdown_connections 2024-09-19T07:31:29.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:29.000+0000 7f85e9000640 1 -- 172.21.15.159:0/2570690024 wait complete. 2024-09-19T07:31:29.867 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T07:31:29.867 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-19T07:31:29.868 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T07:31:30.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:29 smithi189 ceph-mon[34851]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:30.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:29 smithi159 ceph-mon[30472]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:30.238 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:30.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:29 smithi203 ceph-mon[35335]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:30.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.884+0000 7ff00e200640 1 -- 172.21.15.159:0/4013287883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0081079c0 msgr2=0x7ff008107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:30.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.884+0000 7ff00e200640 1 --2- 172.21.15.159:0/4013287883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0081079c0 0x7ff008107da0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feffc009930 tx=0x7feffc02a420 comp rx=0 tx=0).stop 2024-09-19T07:31:30.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.884+0000 7ff00e200640 1 -- 172.21.15.159:0/4013287883 shutdown_connections 2024-09-19T07:31:30.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.884+0000 7ff00e200640 1 --2- 172.21.15.159:0/4013287883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff008102360 0x7ff00810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:30.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.884+0000 7ff00e200640 1 --2- 172.21.15.159:0/4013287883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0081019c0 0x7ff008101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:30.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.884+0000 7ff00e200640 1 --2- 172.21.15.159:0/4013287883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0081079c0 0x7ff008107da0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:30.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.884+0000 7ff00e200640 1 -- 172.21.15.159:0/4013287883 >> 172.21.15.159:0/4013287883 conn(0x7ff0080fd740 msgr2=0x7ff0080ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:30.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.884+0000 7ff00e200640 1 -- 172.21.15.159:0/4013287883 shutdown_connections 2024-09-19T07:31:30.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.884+0000 7ff00e200640 1 -- 172.21.15.159:0/4013287883 wait complete. 2024-09-19T07:31:30.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.885+0000 7ff00e200640 1 Processor -- start 2024-09-19T07:31:30.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.885+0000 7ff00e200640 1 -- start start 2024-09-19T07:31:30.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.885+0000 7ff00e200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0081019c0 0x7ff0081982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:30.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.886+0000 7ff00e200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff008102360 0x7ff0081987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:30.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.886+0000 7ff00e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0081079c0 0x7ff00819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:30.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.886+0000 7ff00e200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff008111e10 con 0x7ff0081079c0 2024-09-19T07:31:30.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.886+0000 7ff00e200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff008111c90 con 0x7ff0081019c0 2024-09-19T07:31:30.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.886+0000 7ff00e200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff008111f90 con 0x7ff008102360 2024-09-19T07:31:30.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.886+0000 7ff00d800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0081079c0 0x7ff00819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:30.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.886+0000 7ff007e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff008102360 0x7ff0081987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:30.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.886+0000 7ff00d800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0081079c0 0x7ff00819cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59936/0 (socket says 172.21.15.159:59936) 2024-09-19T07:31:30.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.886+0000 7ff00d800640 1 -- 172.21.15.159:0/3974552506 learned_addr learned my addr 172.21.15.159:0/3974552506 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:30.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.886+0000 7ff00ce00640 1 --2- 172.21.15.159:0/3974552506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0081019c0 0x7ff0081982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:30.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.886+0000 7ff00d800640 1 -- 172.21.15.159:0/3974552506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff008102360 msgr2=0x7ff0081987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:30.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.886+0000 7ff00d800640 1 --2- 172.21.15.159:0/3974552506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff008102360 0x7ff0081987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:30.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.886+0000 7ff00d800640 1 -- 172.21.15.159:0/3974552506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0081019c0 msgr2=0x7ff0081982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:30.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.886+0000 7ff00d800640 1 --2- 172.21.15.159:0/3974552506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0081019c0 0x7ff0081982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:30.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.886+0000 7ff00d800640 1 -- 172.21.15.159:0/3974552506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00819d260 con 0x7ff0081079c0 2024-09-19T07:31:30.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.887+0000 7ff00d800640 1 --2- 172.21.15.159:0/3974552506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0081079c0 0x7ff00819cb80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff00000eb10 tx=0x7ff00000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:30.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.887+0000 7ff005600640 1 -- 172.21.15.159:0/3974552506 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff000004b20 con 0x7ff0081079c0 2024-09-19T07:31:30.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.887+0000 7ff00e200640 1 -- 172.21.15.159:0/3974552506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00819d550 con 0x7ff0081079c0 2024-09-19T07:31:30.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.887+0000 7ff005600640 1 -- 172.21.15.159:0/3974552506 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff000004cc0 con 0x7ff0081079c0 2024-09-19T07:31:30.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.887+0000 7ff005600640 1 -- 172.21.15.159:0/3974552506 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff000010760 con 0x7ff0081079c0 2024-09-19T07:31:30.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.887+0000 7ff00e200640 1 -- 172.21.15.159:0/3974552506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0081a4e30 con 0x7ff0081079c0 2024-09-19T07:31:30.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.888+0000 7ff00e200640 1 -- 172.21.15.159:0/3974552506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff008101e20 con 0x7ff0081079c0 2024-09-19T07:31:30.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.890+0000 7ff005600640 1 -- 172.21.15.159:0/3974552506 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff000005020 con 0x7ff0081079c0 2024-09-19T07:31:30.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.891+0000 7ff005600640 1 --2- 172.21.15.159:0/3974552506 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feff00a3240 0x7feff00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:30.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.891+0000 7ff005600640 1 -- 172.21.15.159:0/3974552506 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7ff0000d01c0 con 0x7ff0081079c0 2024-09-19T07:31:30.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.891+0000 7ff00ce00640 1 --2- 172.21.15.159:0/3974552506 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feff00a3240 0x7feff00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:30.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.892+0000 7ff00ce00640 1 --2- 172.21.15.159:0/3974552506 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feff00a3240 0x7feff00a5700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feffc006010 tx=0x7feffc000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:30.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:30.895+0000 7ff005600640 1 -- 172.21.15.159:0/3974552506 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff00009cb20 con 0x7ff0081079c0 2024-09-19T07:31:31.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-19T07:31:31.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-09-19T07:31:31.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-19T07:31:31.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-19T07:31:31.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-19T07:31:31.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-19T07:31:31.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]: dispatch 2024-09-19T07:31:31.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-09-19T07:31:31.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-09-19T07:31:31.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-09-19T07:31:31.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-19T07:31:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-09-19T07:31:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-19T07:31:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-19T07:31:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-19T07:31:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-19T07:31:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]: dispatch 2024-09-19T07:31:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-09-19T07:31:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-09-19T07:31:31.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-09-19T07:31:31.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:31.106+0000 7ff00e200640 1 -- 172.21.15.159:0/3974552506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff00802d7c0 con 0x7ff0081079c0 2024-09-19T07:31:31.107 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-19T07:31:31.107 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-09-19T07:31:31.107 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-19T07:31:31.107 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-19T07:31:31.108 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-19T07:31:31.108 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-19T07:31:31.108 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]: dispatch 2024-09-19T07:31:31.108 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-09-19T07:31:31.108 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-09-19T07:31:31.108 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-09-19T07:31:31.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:31.106+0000 7ff005600640 1 -- 172.21.15.159:0/3974552506 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7ff00009c420 con 0x7ff0081079c0 2024-09-19T07:31:31.108 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-19T07:31:31.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:31.109+0000 7ff00e200640 1 -- 172.21.15.159:0/3974552506 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feff00a3240 msgr2=0x7feff00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:31.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:31.109+0000 7ff00e200640 1 --2- 172.21.15.159:0/3974552506 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feff00a3240 0x7feff00a5700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feffc006010 tx=0x7feffc000bc0 comp rx=0 tx=0).stop 2024-09-19T07:31:31.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:31.109+0000 7ff00e200640 1 -- 172.21.15.159:0/3974552506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0081079c0 msgr2=0x7ff00819cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:31.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:31.109+0000 7ff00e200640 1 --2- 172.21.15.159:0/3974552506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0081079c0 0x7ff00819cb80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff00000eb10 tx=0x7ff00000c2d0 comp rx=0 tx=0).stop 2024-09-19T07:31:31.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:31.109+0000 7ff00e200640 1 -- 172.21.15.159:0/3974552506 shutdown_connections 2024-09-19T07:31:31.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:31.109+0000 7ff00e200640 1 --2- 172.21.15.159:0/3974552506 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feff00a3240 0x7feff00a5700 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:31.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:31.109+0000 7ff00e200640 1 --2- 172.21.15.159:0/3974552506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0081079c0 0x7ff00819cb80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:31.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:31.109+0000 7ff00e200640 1 --2- 172.21.15.159:0/3974552506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff008102360 0x7ff0081987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:31.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:31.109+0000 7ff00e200640 1 --2- 172.21.15.159:0/3974552506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0081019c0 0x7ff0081982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:31.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:31.109+0000 7ff00e200640 1 -- 172.21.15.159:0/3974552506 >> 172.21.15.159:0/3974552506 conn(0x7ff0080fd740 msgr2=0x7ff0080ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:31.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:31.109+0000 7ff00e200640 1 -- 172.21.15.159:0/3974552506 shutdown_connections 2024-09-19T07:31:31.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:31.109+0000 7ff00e200640 1 -- 172.21.15.159:0/3974552506 wait complete. 2024-09-19T07:31:31.934 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-19T07:31:31.934 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:31:31.934 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-09-19T07:31:31.934 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-19T07:31:31.934 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-19T07:31:31.934 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-19T07:31:31.935 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:31:31.935 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:31:31.935 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-19T07:31:31.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:31:31.935 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-09-19T07:31:31.971 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:31:31.971 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:31:31.971 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:31:31.972 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:31:31.972 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-09-19T07:31:31.972 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:31:31.972 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:31:31.972 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:31:31.972 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-09-19T07:31:31.972 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:31:31.972 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:31:31.972 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-09-19T07:31:31.973 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:31:31.973 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-09-19T07:31:31.973 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:31:31.973 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-09-19T07:31:31.973 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T07:31:31.973 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 42157sec preferred_lft 42157sec 2024-09-19T07:31:31.973 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link noprefixroute 2024-09-19T07:31:31.973 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:31:31.974 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T07:31:31.974 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:31:31.974 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-09-19T07:31:31.974 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link add name ceph-brx type bridge 2024-09-19T07:31:31.974 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip addr flush dev ceph-brx 2024-09-19T07:31:31.974 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set ceph-brx up 2024-09-19T07:31:31.974 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T07:31:31.974 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-09-19T07:31:32.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:31:32.108 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:31 smithi159 ceph-mon[30472]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:32.108 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-19T07:31:32.108 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-09-19T07:31:32.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-19T07:31:32.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-19T07:31:32.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-19T07:31:32.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-19T07:31:32.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]': finished 2024-09-19T07:31:32.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-19T07:31:32.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-09-19T07:31:32.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-09-19T07:31:32.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:31 smithi159 ceph-mon[30472]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T07:31:32.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:31 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3974552506' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T07:31:32.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:31 smithi203 ceph-mon[35335]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:32.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-19T07:31:32.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-09-19T07:31:32.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-19T07:31:32.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-19T07:31:32.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-19T07:31:32.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-19T07:31:32.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]': finished 2024-09-19T07:31:32.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-19T07:31:32.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-09-19T07:31:32.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-09-19T07:31:32.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:31 smithi203 ceph-mon[35335]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T07:31:32.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:31 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3974552506' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T07:31:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:31 smithi189 ceph-mon[34851]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:31:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-19T07:31:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-09-19T07:31:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-19T07:31:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-19T07:31:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-19T07:31:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-19T07:31:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]': finished 2024-09-19T07:31:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-19T07:31:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-09-19T07:31:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-09-19T07:31:32.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:31 smithi189 ceph-mon[34851]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T07:31:32.307 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:31 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3974552506' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T07:31:32.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:31:32.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:31:32.425 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:31:32.482 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-09-19T07:31:32.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:31:32.486 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-09-19T07:31:32.539 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-09-19T07:31:32.539 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:31:32.539 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T07:31:32.539 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T07:31:32.539 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:31:32.540 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:31:32.540 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-09-19T07:31:32.540 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T07:31:32.540 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T07:31:32.540 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T07:31:32.540 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-09-19T07:31:32.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:31:32.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:32 smithi159 ceph-mon[30472]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T07:31:33.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:31:33.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:31:33.149 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-09-19T07:31:33.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:31:33.186 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list-id 2024-09-19T07:31:33.261 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:31:33.261 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-09-19T07:31:33.261 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T07:31:33.261 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-19T07:31:33.261 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-09-19T07:31:33.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:32 smithi203 ceph-mon[35335]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T07:31:33.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:32 smithi189 ceph-mon[34851]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T07:31:33.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:31:33.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:31:33.440 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-19T07:31:33.441 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:31:33.441 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-09-19T07:31:33.441 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-19T07:31:33.441 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T07:31:33.441 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-19T07:31:33.441 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-19T07:31:33.441 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-19T07:31:33.441 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-09-19T07:31:33.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:31:33.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:31:33.685 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:31:33.685 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-09-19T07:31:33.685 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-09-19T07:31:33.685 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T07:31:33.685 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-09-19T07:31:33.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:31:33.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:31:33.854 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-19T07:31:33.854 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:31:33.854 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-19T07:31:33.889 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-19T07:31:33.890 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:31:33.890 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-19T07:31:33.966 DEBUG:teuthology.orchestra.run.smithi159:> sudo modprobe fuse 2024-09-19T07:31:34.053 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:33 smithi159 ceph-mon[30472]: pgmap v329: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:34.064 DEBUG:teuthology.orchestra.run.smithi159:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:31:34.101 INFO:teuthology.orchestra.run.smithi159.stdout:/proc 2024-09-19T07:31:34.101 INFO:teuthology.orchestra.run.smithi159.stdout:/sys 2024-09-19T07:31:34.101 INFO:teuthology.orchestra.run.smithi159.stdout:/dev 2024-09-19T07:31:34.101 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/kernel/security 2024-09-19T07:31:34.101 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/shm 2024-09-19T07:31:34.101 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/pts 2024-09-19T07:31:34.101 INFO:teuthology.orchestra.run.smithi159.stdout:/run 2024-09-19T07:31:34.101 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/fs/cgroup 2024-09-19T07:31:34.101 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/fs/pstore 2024-09-19T07:31:34.101 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/fs/bpf 2024-09-19T07:31:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:/ 2024-09-19T07:31:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/fs/selinux 2024-09-19T07:31:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:31:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/hugepages 2024-09-19T07:31:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/mqueue 2024-09-19T07:31:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/kernel/debug 2024-09-19T07:31:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/kernel/tracing 2024-09-19T07:31:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/fs/fuse/connections 2024-09-19T07:31:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/kernel/config 2024-09-19T07:31:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:31:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:31:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:31:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:31:34.102 INFO:teuthology.orchestra.run.smithi159.stdout:/run/user/1000 2024-09-19T07:31:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/ceph 2024-09-19T07:31:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay 2024-09-19T07:31:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/8088949d2b7c34de3741ea187f25fcba7cd28bdb1daa395a1f7182ce8d5ff366/merged 2024-09-19T07:31:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/bf84ed47958eed7510913989f7b0e3d0af9f381c7a436741035f560b2cacb7a3/merged 2024-09-19T07:31:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:/run/user/0 2024-09-19T07:31:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:31:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/b063a12dec08c99ce46505d7fbf0619d6bce76ecc72704e9ca727877784fdca3/merged 2024-09-19T07:31:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/784e0670c37957fa47e050345a49c827d7a38db229661c9365bab09f65db6193/merged 2024-09-19T07:31:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/bb714a2ac3a91651741e410e4653cd6c0d653e2466d71881c558cd0ad4bdc7f1/merged 2024-09-19T07:31:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/bcb26abdc704263cdbf263525b1d1764505bd73851b0281f49161647fa8eb3bd/merged 2024-09-19T07:31:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/d6c3a8c80b7c6a4a86d3219d35476a2d3d8f6ec08dea060e4747664f5ac4c26f/merged 2024-09-19T07:31:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/0009c2c2cef3658c3a75541574285d0d4c9c6f9cd44464422712dfa5f920a1d2/merged 2024-09-19T07:31:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/8591686ebc59b9ce936106ebabea3ba443cb2c90887c7c369cf2d44e461066d3/merged 2024-09-19T07:31:34.103 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/34e017d57325247f3dd675f70493866d65e46cc5faf8c604ddc7806c128f1985/merged 2024-09-19T07:31:34.104 INFO:teuthology.orchestra.run.smithi159.stdout:/run/netns 2024-09-19T07:31:34.104 INFO:teuthology.orchestra.run.smithi159.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T07:31:34.104 INFO:teuthology.orchestra.run.smithi159.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T07:31:34.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:31:34.105 DEBUG:teuthology.orchestra.run.smithi159:> ls /sys/fs/fuse/connections 2024-09-19T07:31:34.181 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T07:31:34.181 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-19T07:31:34.226 DEBUG:teuthology.orchestra.run.smithi159:> sudo modprobe fuse 2024-09-19T07:31:34.282 DEBUG:teuthology.orchestra.run.smithi159:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:31:34.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:33 smithi203 ceph-mon[35335]: pgmap v329: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:34.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:33 smithi189 ceph-mon[34851]: pgmap v329: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:34.321 INFO:teuthology.orchestra.run.smithi159.stdout:/proc 2024-09-19T07:31:34.321 INFO:teuthology.orchestra.run.smithi159.stdout:/sys 2024-09-19T07:31:34.321 INFO:teuthology.orchestra.run.smithi159.stdout:/dev 2024-09-19T07:31:34.321 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/kernel/security 2024-09-19T07:31:34.321 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/shm 2024-09-19T07:31:34.321 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/pts 2024-09-19T07:31:34.321 INFO:teuthology.orchestra.run.smithi159.stdout:/run 2024-09-19T07:31:34.322 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/fs/cgroup 2024-09-19T07:31:34.322 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/fs/pstore 2024-09-19T07:31:34.322 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/fs/bpf 2024-09-19T07:31:34.322 INFO:teuthology.orchestra.run.smithi159.stdout:/ 2024-09-19T07:31:34.322 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/fs/selinux 2024-09-19T07:31:34.322 INFO:teuthology.orchestra.run.smithi159.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:31:34.322 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/hugepages 2024-09-19T07:31:34.322 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/mqueue 2024-09-19T07:31:34.322 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/kernel/debug 2024-09-19T07:31:34.322 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/kernel/tracing 2024-09-19T07:31:34.322 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/fs/fuse/connections 2024-09-19T07:31:34.322 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/kernel/config 2024-09-19T07:31:34.322 INFO:teuthology.orchestra.run.smithi159.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:31:34.323 INFO:teuthology.orchestra.run.smithi159.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:31:34.323 INFO:teuthology.orchestra.run.smithi159.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:31:34.323 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:31:34.323 INFO:teuthology.orchestra.run.smithi159.stdout:/run/user/1000 2024-09-19T07:31:34.323 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/ceph 2024-09-19T07:31:34.323 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay 2024-09-19T07:31:34.323 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/8088949d2b7c34de3741ea187f25fcba7cd28bdb1daa395a1f7182ce8d5ff366/merged 2024-09-19T07:31:34.323 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/bf84ed47958eed7510913989f7b0e3d0af9f381c7a436741035f560b2cacb7a3/merged 2024-09-19T07:31:34.323 INFO:teuthology.orchestra.run.smithi159.stdout:/run/user/0 2024-09-19T07:31:34.323 INFO:teuthology.orchestra.run.smithi159.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:31:34.323 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/b063a12dec08c99ce46505d7fbf0619d6bce76ecc72704e9ca727877784fdca3/merged 2024-09-19T07:31:34.323 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/784e0670c37957fa47e050345a49c827d7a38db229661c9365bab09f65db6193/merged 2024-09-19T07:31:34.323 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/bb714a2ac3a91651741e410e4653cd6c0d653e2466d71881c558cd0ad4bdc7f1/merged 2024-09-19T07:31:34.324 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/bcb26abdc704263cdbf263525b1d1764505bd73851b0281f49161647fa8eb3bd/merged 2024-09-19T07:31:34.324 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/d6c3a8c80b7c6a4a86d3219d35476a2d3d8f6ec08dea060e4747664f5ac4c26f/merged 2024-09-19T07:31:34.324 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/0009c2c2cef3658c3a75541574285d0d4c9c6f9cd44464422712dfa5f920a1d2/merged 2024-09-19T07:31:34.324 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/8591686ebc59b9ce936106ebabea3ba443cb2c90887c7c369cf2d44e461066d3/merged 2024-09-19T07:31:34.324 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/34e017d57325247f3dd675f70493866d65e46cc5faf8c604ddc7806c128f1985/merged 2024-09-19T07:31:34.324 INFO:teuthology.orchestra.run.smithi159.stdout:/run/netns 2024-09-19T07:31:34.324 INFO:teuthology.orchestra.run.smithi159.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T07:31:34.324 INFO:teuthology.orchestra.run.smithi159.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T07:31:34.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:31:34.325 DEBUG:teuthology.orchestra.run.smithi159:> ls /sys/fs/fuse/connections 2024-09-19T07:31:34.425 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi159.stderr:2024-09-19T07:31:34.424+0000 7fb811880480 -1 init, newargv = 0x55a65c595370 newargc=15 2024-09-19T07:31:34.425 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi159.stderr:ceph-fuse[74867]: starting ceph client 2024-09-19T07:31:34.437 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi159.stderr:ceph-fuse[74867]: starting fuse 2024-09-19T07:31:35.504 DEBUG:teuthology.orchestra.run.smithi159:> sudo modprobe fuse 2024-09-19T07:31:35.568 DEBUG:teuthology.orchestra.run.smithi159:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T07:31:35.606 INFO:teuthology.orchestra.run.smithi159.stdout:/proc 2024-09-19T07:31:35.606 INFO:teuthology.orchestra.run.smithi159.stdout:/sys 2024-09-19T07:31:35.606 INFO:teuthology.orchestra.run.smithi159.stdout:/dev 2024-09-19T07:31:35.606 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/kernel/security 2024-09-19T07:31:35.606 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/shm 2024-09-19T07:31:35.606 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/pts 2024-09-19T07:31:35.606 INFO:teuthology.orchestra.run.smithi159.stdout:/run 2024-09-19T07:31:35.606 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/fs/cgroup 2024-09-19T07:31:35.606 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/fs/pstore 2024-09-19T07:31:35.606 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/fs/bpf 2024-09-19T07:31:35.606 INFO:teuthology.orchestra.run.smithi159.stdout:/ 2024-09-19T07:31:35.606 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/fs/selinux 2024-09-19T07:31:35.607 INFO:teuthology.orchestra.run.smithi159.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:31:35.607 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/hugepages 2024-09-19T07:31:35.607 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/mqueue 2024-09-19T07:31:35.607 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/kernel/debug 2024-09-19T07:31:35.607 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/kernel/tracing 2024-09-19T07:31:35.607 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/fs/fuse/connections 2024-09-19T07:31:35.607 INFO:teuthology.orchestra.run.smithi159.stdout:/sys/kernel/config 2024-09-19T07:31:35.607 INFO:teuthology.orchestra.run.smithi159.stdout:/run/credentials/systemd-sysctl.service 2024-09-19T07:31:35.607 INFO:teuthology.orchestra.run.smithi159.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-19T07:31:35.607 INFO:teuthology.orchestra.run.smithi159.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-19T07:31:35.607 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/nfs/rpc_pipefs 2024-09-19T07:31:35.607 INFO:teuthology.orchestra.run.smithi159.stdout:/run/user/1000 2024-09-19T07:31:35.607 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/ceph 2024-09-19T07:31:35.607 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay 2024-09-19T07:31:35.608 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/8088949d2b7c34de3741ea187f25fcba7cd28bdb1daa395a1f7182ce8d5ff366/merged 2024-09-19T07:31:35.608 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/bf84ed47958eed7510913989f7b0e3d0af9f381c7a436741035f560b2cacb7a3/merged 2024-09-19T07:31:35.608 INFO:teuthology.orchestra.run.smithi159.stdout:/run/user/0 2024-09-19T07:31:35.608 INFO:teuthology.orchestra.run.smithi159.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T07:31:35.608 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/b063a12dec08c99ce46505d7fbf0619d6bce76ecc72704e9ca727877784fdca3/merged 2024-09-19T07:31:35.608 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/784e0670c37957fa47e050345a49c827d7a38db229661c9365bab09f65db6193/merged 2024-09-19T07:31:35.608 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/bb714a2ac3a91651741e410e4653cd6c0d653e2466d71881c558cd0ad4bdc7f1/merged 2024-09-19T07:31:35.608 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/bcb26abdc704263cdbf263525b1d1764505bd73851b0281f49161647fa8eb3bd/merged 2024-09-19T07:31:35.608 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/d6c3a8c80b7c6a4a86d3219d35476a2d3d8f6ec08dea060e4747664f5ac4c26f/merged 2024-09-19T07:31:35.608 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/0009c2c2cef3658c3a75541574285d0d4c9c6f9cd44464422712dfa5f920a1d2/merged 2024-09-19T07:31:35.608 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/8591686ebc59b9ce936106ebabea3ba443cb2c90887c7c369cf2d44e461066d3/merged 2024-09-19T07:31:35.608 INFO:teuthology.orchestra.run.smithi159.stdout:/var/lib/containers/storage/overlay/34e017d57325247f3dd675f70493866d65e46cc5faf8c604ddc7806c128f1985/merged 2024-09-19T07:31:35.608 INFO:teuthology.orchestra.run.smithi159.stdout:/run/netns 2024-09-19T07:31:35.609 INFO:teuthology.orchestra.run.smithi159.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T07:31:35.609 INFO:teuthology.orchestra.run.smithi159.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T07:31:35.609 INFO:teuthology.orchestra.run.smithi159.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-19T07:31:35.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:31:35.610 DEBUG:teuthology.orchestra.run.smithi159:> ls /sys/fs/fuse/connections 2024-09-19T07:31:35.685 INFO:teuthology.orchestra.run.smithi159.stdout:120 2024-09-19T07:31:35.685 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-19T07:31:35.685 DEBUG:teuthology.orchestra.run.smithi159:> sudo stdin-killer -- python3 -c ' 2024-09-19T07:31:35.685 DEBUG:teuthology.orchestra.run.smithi159:> import glob 2024-09-19T07:31:35.686 DEBUG:teuthology.orchestra.run.smithi159:> import re 2024-09-19T07:31:35.686 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-09-19T07:31:35.686 DEBUG:teuthology.orchestra.run.smithi159:> import subprocess 2024-09-19T07:31:35.686 DEBUG:teuthology.orchestra.run.smithi159:> 2024-09-19T07:31:35.686 DEBUG:teuthology.orchestra.run.smithi159:> def _find_admin_socket(client_name): 2024-09-19T07:31:35.686 DEBUG:teuthology.orchestra.run.smithi159:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-19T07:31:35.686 DEBUG:teuthology.orchestra.run.smithi159:> files = glob.glob(asok_path) 2024-09-19T07:31:35.686 DEBUG:teuthology.orchestra.run.smithi159:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-19T07:31:35.686 DEBUG:teuthology.orchestra.run.smithi159:> 2024-09-19T07:31:35.686 DEBUG:teuthology.orchestra.run.smithi159:> # Given a non-glob path, it better be there 2024-09-19T07:31:35.686 DEBUG:teuthology.orchestra.run.smithi159:> if "*" not in asok_path: 2024-09-19T07:31:35.686 DEBUG:teuthology.orchestra.run.smithi159:> assert(len(files) == 1) 2024-09-19T07:31:35.686 DEBUG:teuthology.orchestra.run.smithi159:> return files[0] 2024-09-19T07:31:35.686 DEBUG:teuthology.orchestra.run.smithi159:> 2024-09-19T07:31:35.687 DEBUG:teuthology.orchestra.run.smithi159:> for f in files: 2024-09-19T07:31:35.687 DEBUG:teuthology.orchestra.run.smithi159:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T07:31:35.687 DEBUG:teuthology.orchestra.run.smithi159:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T07:31:35.687 DEBUG:teuthology.orchestra.run.smithi159:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T07:31:35.687 DEBUG:teuthology.orchestra.run.smithi159:> contents = proc_f.read() 2024-09-19T07:31:35.687 DEBUG:teuthology.orchestra.run.smithi159:> if mountpoint in contents: 2024-09-19T07:31:35.687 DEBUG:teuthology.orchestra.run.smithi159:> return f 2024-09-19T07:31:35.687 DEBUG:teuthology.orchestra.run.smithi159:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T07:31:35.687 DEBUG:teuthology.orchestra.run.smithi159:> 2024-09-19T07:31:35.687 DEBUG:teuthology.orchestra.run.smithi159:> print(_find_admin_socket("client.admin")) 2024-09-19T07:31:35.687 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-09-19T07:31:35.848 INFO:teuthology.orchestra.run.smithi159.stdout:/var/run/ceph/ceph-client.admin.74867.asok 2024-09-19T07:31:35.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:31:35.862 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74867.asok 2024-09-19T07:31:35.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:31:35.863 DEBUG:teuthology.orchestra.run.smithi159:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74867.asok status 2024-09-19T07:31:36.049 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-19T07:31:36.049 INFO:teuthology.orchestra.run.smithi159.stdout: "metadata": { 2024-09-19T07:31:36.050 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T07:31:36.050 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:31:36.050 INFO:teuthology.orchestra.run.smithi159.stdout: "entity_id": "admin", 2024-09-19T07:31:36.050 INFO:teuthology.orchestra.run.smithi159.stdout: "hostname": "smithi159", 2024-09-19T07:31:36.050 INFO:teuthology.orchestra.run.smithi159.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-19T07:31:36.050 INFO:teuthology.orchestra.run.smithi159.stdout: "pid": "74867", 2024-09-19T07:31:36.050 INFO:teuthology.orchestra.run.smithi159.stdout: "root": "/" 2024-09-19T07:31:36.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:31:36.050 INFO:teuthology.orchestra.run.smithi159.stdout: "dentry_count": 0, 2024-09-19T07:31:36.050 INFO:teuthology.orchestra.run.smithi159.stdout: "dentry_pinned_count": 0, 2024-09-19T07:31:36.050 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 24664, 2024-09-19T07:31:36.050 INFO:teuthology.orchestra.run.smithi159.stdout: "inst": { 2024-09-19T07:31:36.050 INFO:teuthology.orchestra.run.smithi159.stdout: "name": { 2024-09-19T07:31:36.050 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "client", 2024-09-19T07:31:36.051 INFO:teuthology.orchestra.run.smithi159.stdout: "num": 24664 2024-09-19T07:31:36.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:31:36.051 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": { 2024-09-19T07:31:36.051 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-19T07:31:36.051 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:0", 2024-09-19T07:31:36.051 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2229331281 2024-09-19T07:31:36.051 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:31:36.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:31:36.051 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": { 2024-09-19T07:31:36.051 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-19T07:31:36.051 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:0", 2024-09-19T07:31:36.051 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2229331281 2024-09-19T07:31:36.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:31:36.051 INFO:teuthology.orchestra.run.smithi159.stdout: "inst_str": "client.24664 172.21.15.159:0/2229331281", 2024-09-19T07:31:36.051 INFO:teuthology.orchestra.run.smithi159.stdout: "addr_str": "172.21.15.159:0/2229331281", 2024-09-19T07:31:36.052 INFO:teuthology.orchestra.run.smithi159.stdout: "inode_count": 1, 2024-09-19T07:31:36.052 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_epoch": 17, 2024-09-19T07:31:36.052 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_epoch": 98, 2024-09-19T07:31:36.052 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_epoch_barrier": 0, 2024-09-19T07:31:36.052 INFO:teuthology.orchestra.run.smithi159.stdout: "blocklisted": false, 2024-09-19T07:31:36.052 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs" 2024-09-19T07:31:36.052 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-19T07:31:36.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:31:36.062 DEBUG:teuthology.orchestra.run.smithi159:> stat --file-system '--printf=%T 2024-09-19T07:31:36.062 DEBUG:teuthology.orchestra.run.smithi159:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-19T07:31:36.098 INFO:teuthology.orchestra.run.smithi159.stdout:fuseblk 2024-09-19T07:31:36.098 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-19T07:31:36.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:31:36.098 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-19T07:31:36.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:35 smithi159 ceph-mon[30472]: pgmap v330: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:36.198 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-19T07:31:36.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T07:31:36.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:35 smithi203 ceph-mon[35335]: pgmap v330: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:36.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:35 smithi189 ceph-mon[34851]: pgmap v330: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:36.337 INFO:teuthology.orchestra.run.smithi159.stdout:# file: . 2024-09-19T07:31:36.337 INFO:teuthology.orchestra.run.smithi159.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-19T07:31:36.337 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:31:36.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:31:36.342 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:31:36.342 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:31:36.379 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T07:31:36.793 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:37.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.356+0000 7fe741c00640 1 -- 172.21.15.159:0/2950275788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe73c101de0 msgr2=0x7fe73c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:37.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.356+0000 7fe741c00640 1 --2- 172.21.15.159:0/2950275788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe73c101de0 0x7fe73c1021c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009900 tx=0x7fe72c02a3f0 comp rx=0 tx=0).stop 2024-09-19T07:31:37.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.356+0000 7fe741c00640 1 -- 172.21.15.159:0/2950275788 shutdown_connections 2024-09-19T07:31:37.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.356+0000 7fe741c00640 1 --2- 172.21.15.159:0/2950275788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe73c106f10 0x7fe73c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:37.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.356+0000 7fe741c00640 1 --2- 172.21.15.159:0/2950275788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe73c102790 0x7fe73c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:37.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.356+0000 7fe741c00640 1 --2- 172.21.15.159:0/2950275788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe73c101de0 0x7fe73c1021c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:37.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.356+0000 7fe741c00640 1 -- 172.21.15.159:0/2950275788 >> 172.21.15.159:0/2950275788 conn(0x7fe73c0fd8c0 msgr2=0x7fe73c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:37.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.356+0000 7fe741c00640 1 -- 172.21.15.159:0/2950275788 shutdown_connections 2024-09-19T07:31:37.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.356+0000 7fe741c00640 1 -- 172.21.15.159:0/2950275788 wait complete. 2024-09-19T07:31:37.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.357+0000 7fe741c00640 1 Processor -- start 2024-09-19T07:31:37.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.357+0000 7fe741c00640 1 -- start start 2024-09-19T07:31:37.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.357+0000 7fe741c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe73c101de0 0x7fe73c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:37.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.357+0000 7fe741c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe73c102790 0x7fe73c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:37.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.357+0000 7fe741c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe73c106f10 0x7fe73c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:37.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.357+0000 7fe741c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c111f70 con 0x7fe73c102790 2024-09-19T07:31:37.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.357+0000 7fe741c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c111df0 con 0x7fe73c106f10 2024-09-19T07:31:37.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.357+0000 7fe741c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1120f0 con 0x7fe73c101de0 2024-09-19T07:31:37.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.358+0000 7fe73aa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe73c101de0 0x7fe73c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:37.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.358+0000 7fe73aa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe73c101de0 0x7fe73c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:57374/0 (socket says 172.21.15.159:57374) 2024-09-19T07:31:37.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.358+0000 7fe73aa00640 1 -- 172.21.15.159:0/143254969 learned_addr learned my addr 172.21.15.159:0/143254969 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:37.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.358+0000 7fe732000640 1 --2- 172.21.15.159:0/143254969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe73c102790 0x7fe73c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:37.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.358+0000 7fe73b400640 1 --2- 172.21.15.159:0/143254969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe73c106f10 0x7fe73c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:37.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.358+0000 7fe73aa00640 1 -- 172.21.15.159:0/143254969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe73c106f10 msgr2=0x7fe73c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:37.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.358+0000 7fe73aa00640 1 --2- 172.21.15.159:0/143254969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe73c106f10 0x7fe73c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:37.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.358+0000 7fe73aa00640 1 -- 172.21.15.159:0/143254969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe73c102790 msgr2=0x7fe73c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:37.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.358+0000 7fe73aa00640 1 --2- 172.21.15.159:0/143254969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe73c102790 0x7fe73c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:37.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.358+0000 7fe73aa00640 1 -- 172.21.15.159:0/143254969 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c19d3e0 con 0x7fe73c101de0 2024-09-19T07:31:37.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.358+0000 7fe73aa00640 1 --2- 172.21.15.159:0/143254969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe73c101de0 0x7fe73c198430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe72c006010 tx=0x7fe72c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:37.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.359+0000 7fe733e00640 1 -- 172.21.15.159:0/143254969 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c03f030 con 0x7fe73c101de0 2024-09-19T07:31:37.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.359+0000 7fe733e00640 1 -- 172.21.15.159:0/143254969 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72c004440 con 0x7fe73c101de0 2024-09-19T07:31:37.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.359+0000 7fe741c00640 1 -- 172.21.15.159:0/143254969 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73c19d670 con 0x7fe73c101de0 2024-09-19T07:31:37.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.359+0000 7fe733e00640 1 -- 172.21.15.159:0/143254969 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c035070 con 0x7fe73c101de0 2024-09-19T07:31:37.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.359+0000 7fe741c00640 1 -- 172.21.15.159:0/143254969 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73c1a4fb0 con 0x7fe73c101de0 2024-09-19T07:31:37.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.360+0000 7fe741c00640 1 -- 172.21.15.159:0/143254969 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe73c1032c0 con 0x7fe73c101de0 2024-09-19T07:31:37.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.362+0000 7fe733e00640 1 -- 172.21.15.159:0/143254969 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fe72c042030 con 0x7fe73c101de0 2024-09-19T07:31:37.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.363+0000 7fe733e00640 1 --2- 172.21.15.159:0/143254969 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe7200a3240 0x7fe7200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:37.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.363+0000 7fe733e00640 1 -- 172.21.15.159:0/143254969 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fe72c0f0280 con 0x7fe73c101de0 2024-09-19T07:31:37.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.363+0000 7fe732000640 1 --2- 172.21.15.159:0/143254969 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe7200a3240 0x7fe7200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:37.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.364+0000 7fe732000640 1 --2- 172.21.15.159:0/143254969 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe7200a3240 0x7fe7200a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0fdd50 tx=0x7fe724005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:37.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.366+0000 7fe733e00640 1 -- 172.21.15.159:0/143254969 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe72c0bc590 con 0x7fe73c101de0 2024-09-19T07:31:37.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.553+0000 7fe741c00640 1 -- 172.21.15.159:0/143254969 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe73c1021c0 con 0x7fe73c101de0 2024-09-19T07:31:37.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.553+0000 7fe733e00640 1 -- 172.21.15.159:0/143254969 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v98) ==== 81+0+600 (secure 0 0 0) 0x7fe73c1021c0 con 0x7fe73c101de0 2024-09-19T07:31:37.554 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/2383222735 2024-09-20T07:23:30.596931+0000 2024-09-19T07:31:37.554 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/1561371050 2024-09-20T07:23:30.596931+0000 2024-09-19T07:31:37.554 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6801/2966947684 2024-09-20T07:23:30.596931+0000 2024-09-19T07:31:37.554 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6800/2966947684 2024-09-20T07:23:30.596931+0000 2024-09-19T07:31:37.554 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6801/432893324 2024-09-20T07:23:03.089190+0000 2024-09-19T07:31:37.554 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6800/432893324 2024-09-20T07:23:03.089190+0000 2024-09-19T07:31:37.554 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/2108730247 2024-09-20T07:23:03.089190+0000 2024-09-19T07:31:37.554 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/3393992615 2024-09-20T07:23:30.596931+0000 2024-09-19T07:31:37.554 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/1366863724 2024-09-20T07:23:03.089190+0000 2024-09-19T07:31:37.555 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/1628344860 2024-09-20T07:23:03.089190+0000 2024-09-19T07:31:37.555 INFO:teuthology.orchestra.run.smithi159.stderr:listed 10 entries 2024-09-19T07:31:37.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.556+0000 7fe741c00640 1 -- 172.21.15.159:0/143254969 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe7200a3240 msgr2=0x7fe7200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:37.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.556+0000 7fe741c00640 1 --2- 172.21.15.159:0/143254969 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe7200a3240 0x7fe7200a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0fdd50 tx=0x7fe724005f90 comp rx=0 tx=0).stop 2024-09-19T07:31:37.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.556+0000 7fe741c00640 1 -- 172.21.15.159:0/143254969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe73c101de0 msgr2=0x7fe73c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:37.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.556+0000 7fe741c00640 1 --2- 172.21.15.159:0/143254969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe73c101de0 0x7fe73c198430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe72c006010 tx=0x7fe72c004040 comp rx=0 tx=0).stop 2024-09-19T07:31:37.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.556+0000 7fe741c00640 1 -- 172.21.15.159:0/143254969 shutdown_connections 2024-09-19T07:31:37.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.556+0000 7fe741c00640 1 --2- 172.21.15.159:0/143254969 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fe7200a3240 0x7fe7200a5700 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:37.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.556+0000 7fe741c00640 1 --2- 172.21.15.159:0/143254969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe73c106f10 0x7fe73c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:37.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.556+0000 7fe741c00640 1 --2- 172.21.15.159:0/143254969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe73c102790 0x7fe73c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:37.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.556+0000 7fe741c00640 1 --2- 172.21.15.159:0/143254969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe73c101de0 0x7fe73c198430 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:37.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.556+0000 7fe741c00640 1 -- 172.21.15.159:0/143254969 >> 172.21.15.159:0/143254969 conn(0x7fe73c0fd8c0 msgr2=0x7fe73c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:37.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.556+0000 7fe741c00640 1 -- 172.21.15.159:0/143254969 shutdown_connections 2024-09-19T07:31:37.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:37.556+0000 7fe741c00640 1 -- 172.21.15.159:0/143254969 wait complete. 2024-09-19T07:31:38.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:37 smithi159 ceph-mon[30472]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:38.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:37 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/143254969' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T07:31:38.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:37 smithi203 ceph-mon[35335]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:38.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:37 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/143254969' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T07:31:38.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:37 smithi189 ceph-mon[34851]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:38.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:37 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/143254969' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T07:31:38.431 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:31:38.431 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:31:38.468 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T07:31:38.878 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:39.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.657+0000 7fb986400640 1 -- 172.21.15.159:0/1284667486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb980101150 msgr2=0x7fb98010de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:39.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.657+0000 7fb986400640 1 --2- 172.21.15.159:0/1284667486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb980101150 0x7fb98010de10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009900 tx=0x7fb96c02a3f0 comp rx=0 tx=0).stop 2024-09-19T07:31:39.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.657+0000 7fb986400640 1 -- 172.21.15.159:0/1284667486 shutdown_connections 2024-09-19T07:31:39.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.657+0000 7fb986400640 1 --2- 172.21.15.159:0/1284667486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb98010e350 0x7fb980110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:39.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.657+0000 7fb986400640 1 --2- 172.21.15.159:0/1284667486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb980101150 0x7fb98010de10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:39.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.657+0000 7fb986400640 1 --2- 172.21.15.159:0/1284667486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9801007a0 0x7fb980100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:39.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.657+0000 7fb986400640 1 -- 172.21.15.159:0/1284667486 >> 172.21.15.159:0/1284667486 conn(0x7fb9800fc630 msgr2=0x7fb9800fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:39.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.657+0000 7fb986400640 1 -- 172.21.15.159:0/1284667486 shutdown_connections 2024-09-19T07:31:39.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.657+0000 7fb986400640 1 -- 172.21.15.159:0/1284667486 wait complete. 2024-09-19T07:31:39.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.658+0000 7fb986400640 1 Processor -- start 2024-09-19T07:31:39.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.659+0000 7fb986400640 1 -- start start 2024-09-19T07:31:39.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.659+0000 7fb986400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9801007a0 0x7fb98019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:39.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.659+0000 7fb986400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb980101150 0x7fb98019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:39.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.659+0000 7fb986400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb98010e350 0x7fb98019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:39.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.659+0000 7fb986400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb980113fd0 con 0x7fb980101150 2024-09-19T07:31:39.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.659+0000 7fb986400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb980113e50 con 0x7fb9801007a0 2024-09-19T07:31:39.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.659+0000 7fb986400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb980114150 con 0x7fb98010e350 2024-09-19T07:31:39.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.660+0000 7fb985000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9801007a0 0x7fb98019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:39.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.660+0000 7fb97fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb980101150 0x7fb98019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:39.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.660+0000 7fb97fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb980101150 0x7fb98019aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:54408/0 (socket says 172.21.15.159:54408) 2024-09-19T07:31:39.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.660+0000 7fb985000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9801007a0 0x7fb98019a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:43594/0 (socket says 172.21.15.159:43594) 2024-09-19T07:31:39.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.660+0000 7fb985000640 1 -- 172.21.15.159:0/4020229930 learned_addr learned my addr 172.21.15.159:0/4020229930 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:39.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.660+0000 7fb985a00640 1 --2- 172.21.15.159:0/4020229930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb98010e350 0x7fb98019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:39.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.660+0000 7fb97fe00640 1 -- 172.21.15.159:0/4020229930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb98010e350 msgr2=0x7fb98019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:39.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.660+0000 7fb97fe00640 1 --2- 172.21.15.159:0/4020229930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb98010e350 0x7fb98019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:39.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.660+0000 7fb97fe00640 1 -- 172.21.15.159:0/4020229930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9801007a0 msgr2=0x7fb98019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:39.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.660+0000 7fb97fe00640 1 --2- 172.21.15.159:0/4020229930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9801007a0 0x7fb98019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:39.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.660+0000 7fb97fe00640 1 -- 172.21.15.159:0/4020229930 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98019f4c0 con 0x7fb980101150 2024-09-19T07:31:39.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.661+0000 7fb97fe00640 1 --2- 172.21.15.159:0/4020229930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb980101150 0x7fb98019aa50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb96c005940 tx=0x7fb96c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:39.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.661+0000 7fb97d600640 1 -- 172.21.15.159:0/4020229930 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c03f030 con 0x7fb980101150 2024-09-19T07:31:39.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.661+0000 7fb97d600640 1 -- 172.21.15.159:0/4020229930 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb96c003ad0 con 0x7fb980101150 2024-09-19T07:31:39.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.661+0000 7fb986400640 1 -- 172.21.15.159:0/4020229930 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98019f750 con 0x7fb980101150 2024-09-19T07:31:39.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.661+0000 7fb97d600640 1 -- 172.21.15.159:0/4020229930 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c035070 con 0x7fb980101150 2024-09-19T07:31:39.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.661+0000 7fb986400640 1 -- 172.21.15.159:0/4020229930 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9801a7090 con 0x7fb980101150 2024-09-19T07:31:39.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.662+0000 7fb986400640 1 -- 172.21.15.159:0/4020229930 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98006aff0 con 0x7fb980101150 2024-09-19T07:31:39.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.664+0000 7fb97d600640 1 -- 172.21.15.159:0/4020229930 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb96c042030 con 0x7fb980101150 2024-09-19T07:31:39.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.665+0000 7fb97d600640 1 --2- 172.21.15.159:0/4020229930 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb9680a3240 0x7fb9680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:39.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.665+0000 7fb97d600640 1 -- 172.21.15.159:0/4020229930 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fb96c0f02a0 con 0x7fb980101150 2024-09-19T07:31:39.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.666+0000 7fb985000640 1 --2- 172.21.15.159:0/4020229930 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb9680a3240 0x7fb9680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:39.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.666+0000 7fb985000640 1 --2- 172.21.15.159:0/4020229930 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb9680a3240 0x7fb9680a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb974009c90 tx=0x7fb974005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:39.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.670+0000 7fb97d600640 1 -- 172.21.15.159:0/4020229930 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb96c0bc5b0 con 0x7fb980101150 2024-09-19T07:31:39.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.861+0000 7fb986400640 1 -- 172.21.15.159:0/4020229930 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb980100b80 con 0x7fb980101150 2024-09-19T07:31:39.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.861+0000 7fb97d600640 1 -- 172.21.15.159:0/4020229930 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v98) ==== 81+0+600 (secure 0 0 0) 0x7fb96c0c08b0 con 0x7fb980101150 2024-09-19T07:31:39.861 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/2383222735 2024-09-20T07:23:30.596931+0000 2024-09-19T07:31:39.861 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/1561371050 2024-09-20T07:23:30.596931+0000 2024-09-19T07:31:39.861 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6801/2966947684 2024-09-20T07:23:30.596931+0000 2024-09-19T07:31:39.862 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6800/2966947684 2024-09-20T07:23:30.596931+0000 2024-09-19T07:31:39.862 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6801/432893324 2024-09-20T07:23:03.089190+0000 2024-09-19T07:31:39.862 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6800/432893324 2024-09-20T07:23:03.089190+0000 2024-09-19T07:31:39.862 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/2108730247 2024-09-20T07:23:03.089190+0000 2024-09-19T07:31:39.862 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/3393992615 2024-09-20T07:23:30.596931+0000 2024-09-19T07:31:39.862 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/1366863724 2024-09-20T07:23:03.089190+0000 2024-09-19T07:31:39.862 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/1628344860 2024-09-20T07:23:03.089190+0000 2024-09-19T07:31:39.862 INFO:teuthology.orchestra.run.smithi159.stderr:listed 10 entries 2024-09-19T07:31:39.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.863+0000 7fb986400640 1 -- 172.21.15.159:0/4020229930 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb9680a3240 msgr2=0x7fb9680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:39.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.863+0000 7fb986400640 1 --2- 172.21.15.159:0/4020229930 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb9680a3240 0x7fb9680a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb974009c90 tx=0x7fb974005f90 comp rx=0 tx=0).stop 2024-09-19T07:31:39.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.863+0000 7fb986400640 1 -- 172.21.15.159:0/4020229930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb980101150 msgr2=0x7fb98019aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:39.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.863+0000 7fb986400640 1 --2- 172.21.15.159:0/4020229930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb980101150 0x7fb98019aa50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb96c005940 tx=0x7fb96c004270 comp rx=0 tx=0).stop 2024-09-19T07:31:39.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.864+0000 7fb986400640 1 -- 172.21.15.159:0/4020229930 shutdown_connections 2024-09-19T07:31:39.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.864+0000 7fb986400640 1 --2- 172.21.15.159:0/4020229930 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb9680a3240 0x7fb9680a5700 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:39.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.864+0000 7fb986400640 1 --2- 172.21.15.159:0/4020229930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb98010e350 0x7fb98019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:39.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.864+0000 7fb986400640 1 --2- 172.21.15.159:0/4020229930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb980101150 0x7fb98019aa50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:39.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.864+0000 7fb986400640 1 --2- 172.21.15.159:0/4020229930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9801007a0 0x7fb98019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:39.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.864+0000 7fb986400640 1 -- 172.21.15.159:0/4020229930 >> 172.21.15.159:0/4020229930 conn(0x7fb9800fc630 msgr2=0x7fb9800fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:39.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.864+0000 7fb986400640 1 -- 172.21.15.159:0/4020229930 shutdown_connections 2024-09-19T07:31:39.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:39.864+0000 7fb986400640 1 -- 172.21.15.159:0/4020229930 wait complete. 2024-09-19T07:31:40.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:39 smithi159 ceph-mon[30472]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:40.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:39 smithi203 ceph-mon[35335]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:40.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:39 smithi189 ceph-mon[34851]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:40.695 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi159.front.sepia.ceph.com... 2024-09-19T07:31:40.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:31:40.696 DEBUG:teuthology.orchestra.run.smithi159:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-19T07:31:40.751 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi159.stderr:ceph-fuse[74867]: fuse finished with error 0 and tester_r 0 2024-09-19T07:31:40.754 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T07:31:40.784 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-09-19T07:31:40.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:31:40.785 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-19T07:31:40.867 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:31:40.955 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:40 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/4020229930' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T07:31:41.278 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:41.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:40 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/4020229930' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T07:31:41.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:40 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/4020229930' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T07:31:41.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.918+0000 7effd4000640 1 -- 172.21.15.159:0/3813868790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effcc074f40 msgr2=0x7effcc10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:41.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.918+0000 7effd4000640 1 --2- 172.21.15.159:0/3813868790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effcc074f40 0x7effcc10de80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7effb8009900 tx=0x7effb802a480 comp rx=0 tx=0).stop 2024-09-19T07:31:41.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.919+0000 7effd4000640 1 -- 172.21.15.159:0/3813868790 shutdown_connections 2024-09-19T07:31:41.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.919+0000 7effd4000640 1 --2- 172.21.15.159:0/3813868790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effcc10e3c0 0x7effcc110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:41.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.919+0000 7effd4000640 1 --2- 172.21.15.159:0/3813868790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effcc074f40 0x7effcc10de80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:41.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.919+0000 7effd4000640 1 --2- 172.21.15.159:0/3813868790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7effcc074620 0x7effcc074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:41.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.919+0000 7effd4000640 1 -- 172.21.15.159:0/3813868790 >> 172.21.15.159:0/3813868790 conn(0x7effcc0fd8c0 msgr2=0x7effcc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:41.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.919+0000 7effd4000640 1 -- 172.21.15.159:0/3813868790 shutdown_connections 2024-09-19T07:31:41.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.919+0000 7effd4000640 1 -- 172.21.15.159:0/3813868790 wait complete. 2024-09-19T07:31:41.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.920+0000 7effd4000640 1 Processor -- start 2024-09-19T07:31:41.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.920+0000 7effd4000640 1 -- start start 2024-09-19T07:31:41.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.920+0000 7effd4000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effcc074620 0x7effcc19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:41.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.920+0000 7effd4000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7effcc074f40 0x7effcc19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:41.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.920+0000 7effd4000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effcc10e3c0 0x7effcc19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:41.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.920+0000 7effd4000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7effcc114140 con 0x7effcc074620 2024-09-19T07:31:41.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.920+0000 7effd4000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7effcc113fc0 con 0x7effcc074f40 2024-09-19T07:31:41.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.920+0000 7effd4000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7effcc1142c0 con 0x7effcc10e3c0 2024-09-19T07:31:41.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.920+0000 7effd1600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effcc074620 0x7effcc19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:41.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.921+0000 7effd1600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effcc074620 0x7effcc19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:54426/0 (socket says 172.21.15.159:54426) 2024-09-19T07:31:41.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.921+0000 7effd1600640 1 -- 172.21.15.159:0/1367327778 learned_addr learned my addr 172.21.15.159:0/1367327778 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:41.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.921+0000 7effd0c00640 1 --2- 172.21.15.159:0/1367327778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7effcc074f40 0x7effcc19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:41.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.921+0000 7effd2000640 1 --2- 172.21.15.159:0/1367327778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effcc10e3c0 0x7effcc19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:41.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.921+0000 7effd1600640 1 -- 172.21.15.159:0/1367327778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effcc10e3c0 msgr2=0x7effcc19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:41.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.921+0000 7effd1600640 1 --2- 172.21.15.159:0/1367327778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effcc10e3c0 0x7effcc19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:41.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.921+0000 7effd1600640 1 -- 172.21.15.159:0/1367327778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7effcc074f40 msgr2=0x7effcc19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:41.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.921+0000 7effd1600640 1 --2- 172.21.15.159:0/1367327778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7effcc074f40 0x7effcc19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:41.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.921+0000 7effd1600640 1 -- 172.21.15.159:0/1367327778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effcc19f640 con 0x7effcc074620 2024-09-19T07:31:41.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.921+0000 7effd1600640 1 --2- 172.21.15.159:0/1367327778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effcc074620 0x7effcc19a690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7effc000e9f0 tx=0x7effc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:41.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.922+0000 7effca000640 1 -- 172.21.15.159:0/1367327778 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc000cd80 con 0x7effcc074620 2024-09-19T07:31:41.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.922+0000 7effca000640 1 -- 172.21.15.159:0/1367327778 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc0010910 con 0x7effcc074620 2024-09-19T07:31:41.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.922+0000 7effd4000640 1 -- 172.21.15.159:0/1367327778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effcc19f930 con 0x7effcc074620 2024-09-19T07:31:41.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.922+0000 7effca000640 1 -- 172.21.15.159:0/1367327778 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0018990 con 0x7effcc074620 2024-09-19T07:31:41.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.922+0000 7effd4000640 1 -- 172.21.15.159:0/1367327778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effcc1a7210 con 0x7effcc074620 2024-09-19T07:31:41.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.923+0000 7effd4000640 1 -- 172.21.15.159:0/1367327778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effcc074a00 con 0x7effcc074620 2024-09-19T07:31:41.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.925+0000 7effca000640 1 -- 172.21.15.159:0/1367327778 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7effc0010ac0 con 0x7effcc074620 2024-09-19T07:31:41.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.926+0000 7effca000640 1 --2- 172.21.15.159:0/1367327778 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7effb40a32c0 0x7effb40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:41.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.926+0000 7effca000640 1 -- 172.21.15.159:0/1367327778 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7effc00cff40 con 0x7effcc074620 2024-09-19T07:31:41.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.926+0000 7effd0c00640 1 --2- 172.21.15.159:0/1367327778 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7effb40a32c0 0x7effb40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:41.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.927+0000 7effd0c00640 1 --2- 172.21.15.159:0/1367327778 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7effb40a32c0 0x7effb40a5780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7effb8009230 tx=0x7effb80058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:41.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:41.930+0000 7effca000640 1 -- 172.21.15.159:0/1367327778 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7effc009c250 con 0x7effcc074620 2024-09-19T07:31:42.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:42.107+0000 7effd4000640 1 -- 172.21.15.159:0/1367327778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7effcc109570 con 0x7effcc074620 2024-09-19T07:31:42.108 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:41 smithi159 ceph-mon[30472]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 206 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:42.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:42.109+0000 7effca000640 1 -- 172.21.15.159:0/1367327778 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24765 (secure 0 0 0) 0x7effc00a0550 con 0x7effcc074620 2024-09-19T07:31:42.110 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:31:42.110 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":98,"fsid":"c9d18048-7657-11ef-bceb-c7b262605968","created":"2024-09-19T07:22:32.059836+0000","modified":"2024-09-19T07:31:31.799299+0000","last_up_change":"2024-09-19T07:30:03.328697+0000","last_in_change":"2024-09-19T07:29:38.987579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:25:55.202457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:30:58.022056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:31:01.042290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:31:10.977289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"752f5466-fe04-408a-8566-86625947fd3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6803","nonce":2738900830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6805","nonce":2738900830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6809","nonce":2738900830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6807","nonce":2738900830}]},"public_addr":"172.21.15.159:6803/2738900830","cluster_addr":"172.21.15.159:6805/2738900830","heartbeat_back_addr":"172.21.15.159:6809/2738900830","heartbeat_front_addr":"172.21.15.159:6807/2738900830","state":["exists","up"]},{"osd":1,"uuid":"7513adda-ca82-4df4-aa99-3b050b410424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6801","nonce":265696178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6803","nonce":265696178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6807","nonce":265696178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6805","nonce":265696178}]},"public_addr":"172.21.15.189:6801/265696178","cluster_addr":"172.21.15.189:6803/265696178","heartbeat_back_addr":"172.21.15.189:6807/265696178","heartbeat_front_addr":"172.21.15.189:6805/265696178","state":["exists","up"]},{"osd":2,"uuid":"aa03be79-58d8-4eb7-a051-0b46c502b12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6801","nonce":1826565288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6803","nonce":1826565288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6807","nonce":1826565288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6805","nonce":1826565288}]},"public_addr":"172.21.15.203:6801/1826565288","cluster_addr":"172.21.15.203:6803/1826565288","heartbeat_back_addr":"172.21.15.203:6807/1826565288","heartbeat_front_addr":"172.21.15.203:6805/1826565288","state":["exists","up"]},{"osd":3,"uuid":"fcd734f1-f40f-4566-82ab-c4607039f23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6811","nonce":109457380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6813","nonce":109457380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6817","nonce":109457380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6815","nonce":109457380}]},"public_addr":"172.21.15.159:6811/109457380","cluster_addr":"172.21.15.159:6813/109457380","heartbeat_back_addr":"172.21.15.159:6817/109457380","heartbeat_front_addr":"172.21.15.159:6815/109457380","state":["exists","up"]},{"osd":4,"uuid":"bc344c9e-2267-4058-bbe1-9c270327db8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6809","nonce":1036888180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6811","nonce":1036888180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6815","nonce":1036888180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6813","nonce":1036888180}]},"public_addr":"172.21.15.189:6809/1036888180","cluster_addr":"172.21.15.189:6811/1036888180","heartbeat_back_addr":"172.21.15.189:6815/1036888180","heartbeat_front_addr":"172.21.15.189:6813/1036888180","state":["exists","up"]},{"osd":5,"uuid":"86c48303-929d-42f4-bf48-cbb32649f95f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6809","nonce":4212601512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6811","nonce":4212601512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6815","nonce":4212601512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6813","nonce":4212601512}]},"public_addr":"172.21.15.203:6809/4212601512","cluster_addr":"172.21.15.203:6811/4212601512","heartbeat_back_addr":"172.21.15.203:6815/4212601512","heartbeat_front_addr":"172.21.15.203:6813/4212601512","state":["exists","up"]},{"osd":6,"uuid":"f0176ed1-c3a7-4d75-b213-fe88fab4441a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6819","nonce":3817684578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6821","nonce":3817684578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6825","nonce":3817684578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6823","nonce":3817684578}]},"public_addr":"172.21.15.159:6819/3817684578","cluster_addr":"172.21.15.159:6821/3817684578","heartbeat_back_addr":"172.21.15.159:6825/3817684578","heartbeat_front_addr":"172.21.15.159:6823/3817684578","state":["exists","up"]},{"osd":7,"uuid":"921bd59d-9415-4559-8e9f-035d336afd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6817","nonce":3936826456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6819","nonce":3936826456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6823","nonce":3936826456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6821","nonce":3936826456}]},"public_addr":"172.21.15.189:6817/3936826456","cluster_addr":"172.21.15.189:6819/3936826456","heartbeat_back_addr":"172.21.15.189:6823/3936826456","heartbeat_front_addr":"172.21.15.189:6821/3936826456","state":["exists","up"]},{"osd":8,"uuid":"ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6817","nonce":4062926645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6819","nonce":4062926645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6823","nonce":4062926645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6821","nonce":4062926645}]},"public_addr":"172.21.15.203:6817/4062926645","cluster_addr":"172.21.15.203:6819/4062926645","heartbeat_back_addr":"172.21.15.203:6823/4062926645","heartbeat_front_addr":"172.21.15.203:6821/4062926645","state":["exists","up"]},{"osd":9,"uuid":"df29a10f-1c60-41cb-9c2d-bc9f1cb174bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6827","nonce":4185848747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6829","nonce":4185848747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6833","nonce":4185848747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6831","nonce":4185848747}]},"public_addr":"172.21.15.159:6827/4185848747","cluster_addr":"172.21.15.159:6829/4185848747","heartbeat_back_addr":"172.21.15.159:6833/4185848747","heartbeat_front_addr":"172.21.15.159:6831/4185848747","state":["exists","up"]},{"osd":10,"uuid":"ccc440b4-4682-4998-bb53-95e24423cd1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6825","nonce":3018377605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6827","nonce":3018377605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6831","nonce":3018377605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6829","nonce":3018377605}]},"public_addr":"172.21.15.189:6825/3018377605","cluster_addr":"172.21.15.189:6827/3018377605","heartbeat_back_addr":"172.21.15.189:6831/3018377605","heartbeat_front_addr":"172.21.15.189:6829/3018377605","state":["exists","up"]},{"osd":11,"uuid":"e84afc5f-8051-4341-8fd0-3afb482ab4df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6825","nonce":2807503891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6827","nonce":2807503891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6831","nonce":2807503891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6829","nonce":2807503891}]},"public_addr":"172.21.15.203:6825/2807503891","cluster_addr":"172.21.15.203:6827/2807503891","heartbeat_back_addr":"172.21.15.203:6831/2807503891","heartbeat_front_addr":"172.21.15.203:6829/2807503891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:00.093677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:25.477205+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:51.345020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:21.077603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:48.822640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:13.516029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:44.382228+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:10.810700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:36.817155+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:07.728553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:34.747540+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:01.249332+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:0/2383222735":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1561371050":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6800/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:6800/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/2108730247":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/3393992615":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1366863724":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/1628344860":"2024-09-20T07:23:03.089190+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:31:42.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:42.112+0000 7effd4000640 1 -- 172.21.15.159:0/1367327778 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7effb40a32c0 msgr2=0x7effb40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:42.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:42.112+0000 7effd4000640 1 --2- 172.21.15.159:0/1367327778 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7effb40a32c0 0x7effb40a5780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7effb8009230 tx=0x7effb80058d0 comp rx=0 tx=0).stop 2024-09-19T07:31:42.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:42.112+0000 7effd4000640 1 -- 172.21.15.159:0/1367327778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effcc074620 msgr2=0x7effcc19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:42.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:42.112+0000 7effd4000640 1 --2- 172.21.15.159:0/1367327778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effcc074620 0x7effcc19a690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7effc000e9f0 tx=0x7effc000eeb0 comp rx=0 tx=0).stop 2024-09-19T07:31:42.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:42.112+0000 7effd4000640 1 -- 172.21.15.159:0/1367327778 shutdown_connections 2024-09-19T07:31:42.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:42.112+0000 7effd4000640 1 --2- 172.21.15.159:0/1367327778 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7effb40a32c0 0x7effb40a5780 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:42.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:42.112+0000 7effd4000640 1 --2- 172.21.15.159:0/1367327778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effcc10e3c0 0x7effcc19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:42.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:42.112+0000 7effd4000640 1 --2- 172.21.15.159:0/1367327778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7effcc074f40 0x7effcc19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:42.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:42.112+0000 7effd4000640 1 --2- 172.21.15.159:0/1367327778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effcc074620 0x7effcc19a690 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:42.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:42.112+0000 7effd4000640 1 -- 172.21.15.159:0/1367327778 >> 172.21.15.159:0/1367327778 conn(0x7effcc0fd8c0 msgr2=0x7effcc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:42.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:42.112+0000 7effd4000640 1 -- 172.21.15.159:0/1367327778 shutdown_connections 2024-09-19T07:31:42.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:42.112+0000 7effd4000640 1 -- 172.21.15.159:0/1367327778 wait complete. 2024-09-19T07:31:42.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:41 smithi203 ceph-mon[35335]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 206 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:42.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:41 smithi189 ceph-mon[34851]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 206 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:42.940 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:31:43.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:42 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1367327778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:31:43.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:42 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1367327778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:31:43.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:42 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1367327778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:31:43.317 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:43.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.934+0000 7f19a4c00640 1 -- 172.21.15.159:0/3539334560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19a0101de0 msgr2=0x7f19a01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:43.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.934+0000 7f19a4c00640 1 --2- 172.21.15.159:0/3539334560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19a0101de0 0x7f19a01021c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1990009900 tx=0x7f199002a410 comp rx=0 tx=0).stop 2024-09-19T07:31:43.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.934+0000 7f19a4c00640 1 -- 172.21.15.159:0/3539334560 shutdown_connections 2024-09-19T07:31:43.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.934+0000 7f19a4c00640 1 --2- 172.21.15.159:0/3539334560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19a0106f10 0x7f19a010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:43.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.934+0000 7f19a4c00640 1 --2- 172.21.15.159:0/3539334560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19a0102790 0x7f19a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:43.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.934+0000 7f19a4c00640 1 --2- 172.21.15.159:0/3539334560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19a0101de0 0x7f19a01021c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:43.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.934+0000 7f19a4c00640 1 -- 172.21.15.159:0/3539334560 >> 172.21.15.159:0/3539334560 conn(0x7f19a00fd8c0 msgr2=0x7f19a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:43.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.934+0000 7f19a4c00640 1 -- 172.21.15.159:0/3539334560 shutdown_connections 2024-09-19T07:31:43.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.934+0000 7f19a4c00640 1 -- 172.21.15.159:0/3539334560 wait complete. 2024-09-19T07:31:43.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.935+0000 7f19a4c00640 1 Processor -- start 2024-09-19T07:31:43.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.935+0000 7f19a4c00640 1 -- start start 2024-09-19T07:31:43.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f19a4c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19a0101de0 0x7f19a0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:43.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f19a4c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19a0102790 0x7f19a0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f19a4c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19a0106f10 0x7f19a019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f19a4c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f19a0111f70 con 0x7f19a0106f10 2024-09-19T07:31:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f19a4c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f19a0111df0 con 0x7f19a0101de0 2024-09-19T07:31:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f19a4c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f19a01120f0 con 0x7f19a0102790 2024-09-19T07:31:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f199e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19a0106f10 0x7f19a019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f199e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19a0106f10 0x7f19a019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51126/0 (socket says 172.21.15.159:51126) 2024-09-19T07:31:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f199e800640 1 -- 172.21.15.159:0/4265636228 learned_addr learned my addr 172.21.15.159:0/4265636228 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f199d400640 1 --2- 172.21.15.159:0/4265636228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19a0102790 0x7f19a0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f199de00640 1 --2- 172.21.15.159:0/4265636228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19a0101de0 0x7f19a0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f199e800640 1 -- 172.21.15.159:0/4265636228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19a0102790 msgr2=0x7f19a0198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:43.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f199e800640 1 --2- 172.21.15.159:0/4265636228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19a0102790 0x7f19a0198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:43.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f199e800640 1 -- 172.21.15.159:0/4265636228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19a0101de0 msgr2=0x7f19a0198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:43.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f199e800640 1 --2- 172.21.15.159:0/4265636228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19a0101de0 0x7f19a0198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:43.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f199e800640 1 -- 172.21.15.159:0/4265636228 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a019d3c0 con 0x7f19a0106f10 2024-09-19T07:31:43.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f199d400640 1 --2- 172.21.15.159:0/4265636228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19a0102790 0x7f19a0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:31:43.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.936+0000 7f199de00640 1 --2- 172.21.15.159:0/4265636228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19a0101de0 0x7f19a0198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:31:43.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.937+0000 7f199e800640 1 --2- 172.21.15.159:0/4265636228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19a0106f10 0x7f19a019cce0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f199800eb10 tx=0x7f199800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:43.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.937+0000 7f1996a00640 1 -- 172.21.15.159:0/4265636228 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998004b20 con 0x7f19a0106f10 2024-09-19T07:31:43.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.937+0000 7f1996a00640 1 -- 172.21.15.159:0/4265636228 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1998004cc0 con 0x7f19a0106f10 2024-09-19T07:31:43.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.937+0000 7f1996a00640 1 -- 172.21.15.159:0/4265636228 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998010760 con 0x7f19a0106f10 2024-09-19T07:31:43.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.937+0000 7f19a4c00640 1 -- 172.21.15.159:0/4265636228 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a019d6b0 con 0x7f19a0106f10 2024-09-19T07:31:43.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.937+0000 7f19a4c00640 1 -- 172.21.15.159:0/4265636228 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a01a4f90 con 0x7f19a0106f10 2024-09-19T07:31:43.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.938+0000 7f19a4c00640 1 -- 172.21.15.159:0/4265636228 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a01021c0 con 0x7f19a0106f10 2024-09-19T07:31:43.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.940+0000 7f1996a00640 1 -- 172.21.15.159:0/4265636228 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f1998005020 con 0x7f19a0106f10 2024-09-19T07:31:43.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.941+0000 7f1996a00640 1 --2- 172.21.15.159:0/4265636228 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f19840a32c0 0x7f19840a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:43.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.941+0000 7f1996a00640 1 -- 172.21.15.159:0/4265636228 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f19980d0160 con 0x7f19a0106f10 2024-09-19T07:31:43.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.942+0000 7f199de00640 1 --2- 172.21.15.159:0/4265636228 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f19840a32c0 0x7f19840a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:43.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.942+0000 7f199de00640 1 --2- 172.21.15.159:0/4265636228 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f19840a32c0 0x7f19840a5780 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1990009230 tx=0x7f19900058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:43.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:43.946+0000 7f1996a00640 1 -- 172.21.15.159:0/4265636228 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f199809cac0 con 0x7f19a0106f10 2024-09-19T07:31:44.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:44.122+0000 7f19a4c00640 1 -- 172.21.15.159:0/4265636228 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19a0109b60 con 0x7f19a0106f10 2024-09-19T07:31:44.122 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:43 smithi159 ceph-mon[30472]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 469 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:44.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:44.123+0000 7f1996a00640 1 -- 172.21.15.159:0/4265636228 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24765 (secure 0 0 0) 0x7f199809c3c0 con 0x7f19a0106f10 2024-09-19T07:31:44.124 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:31:44.124 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":98,"fsid":"c9d18048-7657-11ef-bceb-c7b262605968","created":"2024-09-19T07:22:32.059836+0000","modified":"2024-09-19T07:31:31.799299+0000","last_up_change":"2024-09-19T07:30:03.328697+0000","last_in_change":"2024-09-19T07:29:38.987579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:25:55.202457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:30:58.022056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:31:01.042290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:31:10.977289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"752f5466-fe04-408a-8566-86625947fd3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6803","nonce":2738900830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6805","nonce":2738900830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6809","nonce":2738900830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6807","nonce":2738900830}]},"public_addr":"172.21.15.159:6803/2738900830","cluster_addr":"172.21.15.159:6805/2738900830","heartbeat_back_addr":"172.21.15.159:6809/2738900830","heartbeat_front_addr":"172.21.15.159:6807/2738900830","state":["exists","up"]},{"osd":1,"uuid":"7513adda-ca82-4df4-aa99-3b050b410424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6801","nonce":265696178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6803","nonce":265696178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6807","nonce":265696178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6805","nonce":265696178}]},"public_addr":"172.21.15.189:6801/265696178","cluster_addr":"172.21.15.189:6803/265696178","heartbeat_back_addr":"172.21.15.189:6807/265696178","heartbeat_front_addr":"172.21.15.189:6805/265696178","state":["exists","up"]},{"osd":2,"uuid":"aa03be79-58d8-4eb7-a051-0b46c502b12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6801","nonce":1826565288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6803","nonce":1826565288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6807","nonce":1826565288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6805","nonce":1826565288}]},"public_addr":"172.21.15.203:6801/1826565288","cluster_addr":"172.21.15.203:6803/1826565288","heartbeat_back_addr":"172.21.15.203:6807/1826565288","heartbeat_front_addr":"172.21.15.203:6805/1826565288","state":["exists","up"]},{"osd":3,"uuid":"fcd734f1-f40f-4566-82ab-c4607039f23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6811","nonce":109457380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6813","nonce":109457380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6817","nonce":109457380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6815","nonce":109457380}]},"public_addr":"172.21.15.159:6811/109457380","cluster_addr":"172.21.15.159:6813/109457380","heartbeat_back_addr":"172.21.15.159:6817/109457380","heartbeat_front_addr":"172.21.15.159:6815/109457380","state":["exists","up"]},{"osd":4,"uuid":"bc344c9e-2267-4058-bbe1-9c270327db8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6809","nonce":1036888180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6811","nonce":1036888180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6815","nonce":1036888180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6813","nonce":1036888180}]},"public_addr":"172.21.15.189:6809/1036888180","cluster_addr":"172.21.15.189:6811/1036888180","heartbeat_back_addr":"172.21.15.189:6815/1036888180","heartbeat_front_addr":"172.21.15.189:6813/1036888180","state":["exists","up"]},{"osd":5,"uuid":"86c48303-929d-42f4-bf48-cbb32649f95f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6809","nonce":4212601512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6811","nonce":4212601512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6815","nonce":4212601512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6813","nonce":4212601512}]},"public_addr":"172.21.15.203:6809/4212601512","cluster_addr":"172.21.15.203:6811/4212601512","heartbeat_back_addr":"172.21.15.203:6815/4212601512","heartbeat_front_addr":"172.21.15.203:6813/4212601512","state":["exists","up"]},{"osd":6,"uuid":"f0176ed1-c3a7-4d75-b213-fe88fab4441a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6819","nonce":3817684578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6821","nonce":3817684578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6825","nonce":3817684578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6823","nonce":3817684578}]},"public_addr":"172.21.15.159:6819/3817684578","cluster_addr":"172.21.15.159:6821/3817684578","heartbeat_back_addr":"172.21.15.159:6825/3817684578","heartbeat_front_addr":"172.21.15.159:6823/3817684578","state":["exists","up"]},{"osd":7,"uuid":"921bd59d-9415-4559-8e9f-035d336afd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6817","nonce":3936826456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6819","nonce":3936826456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6823","nonce":3936826456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6821","nonce":3936826456}]},"public_addr":"172.21.15.189:6817/3936826456","cluster_addr":"172.21.15.189:6819/3936826456","heartbeat_back_addr":"172.21.15.189:6823/3936826456","heartbeat_front_addr":"172.21.15.189:6821/3936826456","state":["exists","up"]},{"osd":8,"uuid":"ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6817","nonce":4062926645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6819","nonce":4062926645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6823","nonce":4062926645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6821","nonce":4062926645}]},"public_addr":"172.21.15.203:6817/4062926645","cluster_addr":"172.21.15.203:6819/4062926645","heartbeat_back_addr":"172.21.15.203:6823/4062926645","heartbeat_front_addr":"172.21.15.203:6821/4062926645","state":["exists","up"]},{"osd":9,"uuid":"df29a10f-1c60-41cb-9c2d-bc9f1cb174bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6827","nonce":4185848747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6829","nonce":4185848747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6833","nonce":4185848747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6831","nonce":4185848747}]},"public_addr":"172.21.15.159:6827/4185848747","cluster_addr":"172.21.15.159:6829/4185848747","heartbeat_back_addr":"172.21.15.159:6833/4185848747","heartbeat_front_addr":"172.21.15.159:6831/4185848747","state":["exists","up"]},{"osd":10,"uuid":"ccc440b4-4682-4998-bb53-95e24423cd1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6825","nonce":3018377605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6827","nonce":3018377605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6831","nonce":3018377605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6829","nonce":3018377605}]},"public_addr":"172.21.15.189:6825/3018377605","cluster_addr":"172.21.15.189:6827/3018377605","heartbeat_back_addr":"172.21.15.189:6831/3018377605","heartbeat_front_addr":"172.21.15.189:6829/3018377605","state":["exists","up"]},{"osd":11,"uuid":"e84afc5f-8051-4341-8fd0-3afb482ab4df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6825","nonce":2807503891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6827","nonce":2807503891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6831","nonce":2807503891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6829","nonce":2807503891}]},"public_addr":"172.21.15.203:6825/2807503891","cluster_addr":"172.21.15.203:6827/2807503891","heartbeat_back_addr":"172.21.15.203:6831/2807503891","heartbeat_front_addr":"172.21.15.203:6829/2807503891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:00.093677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:25.477205+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:51.345020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:21.077603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:48.822640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:13.516029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:44.382228+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:10.810700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:36.817155+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:07.728553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:34.747540+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:01.249332+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:0/2383222735":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1561371050":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6800/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:6800/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/2108730247":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/3393992615":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1366863724":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/1628344860":"2024-09-20T07:23:03.089190+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:31:44.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:44.126+0000 7f19a4c00640 1 -- 172.21.15.159:0/4265636228 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f19840a32c0 msgr2=0x7f19840a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:44.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:44.126+0000 7f19a4c00640 1 --2- 172.21.15.159:0/4265636228 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f19840a32c0 0x7f19840a5780 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1990009230 tx=0x7f19900058d0 comp rx=0 tx=0).stop 2024-09-19T07:31:44.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:44.126+0000 7f19a4c00640 1 -- 172.21.15.159:0/4265636228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19a0106f10 msgr2=0x7f19a019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:44.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:44.126+0000 7f19a4c00640 1 --2- 172.21.15.159:0/4265636228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19a0106f10 0x7f19a019cce0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f199800eb10 tx=0x7f199800c2d0 comp rx=0 tx=0).stop 2024-09-19T07:31:44.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:44.126+0000 7f19a4c00640 1 -- 172.21.15.159:0/4265636228 shutdown_connections 2024-09-19T07:31:44.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:44.126+0000 7f19a4c00640 1 --2- 172.21.15.159:0/4265636228 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f19840a32c0 0x7f19840a5780 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:44.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:44.126+0000 7f19a4c00640 1 --2- 172.21.15.159:0/4265636228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19a0106f10 0x7f19a019cce0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:44.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:44.126+0000 7f19a4c00640 1 --2- 172.21.15.159:0/4265636228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19a0102790 0x7f19a0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:44.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:44.126+0000 7f19a4c00640 1 --2- 172.21.15.159:0/4265636228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19a0101de0 0x7f19a0198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:44.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:44.126+0000 7f19a4c00640 1 -- 172.21.15.159:0/4265636228 >> 172.21.15.159:0/4265636228 conn(0x7f19a00fd8c0 msgr2=0x7f19a00fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:44.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:44.126+0000 7f19a4c00640 1 -- 172.21.15.159:0/4265636228 shutdown_connections 2024-09-19T07:31:44.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:44.126+0000 7f19a4c00640 1 -- 172.21.15.159:0/4265636228 wait complete. 2024-09-19T07:31:44.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:43 smithi203 ceph-mon[35335]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 469 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:44.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:43 smithi189 ceph-mon[34851]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 469 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-19T07:31:44.925 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-19T07:31:45.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:44 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/4265636228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:31:45.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:44 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/4265636228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:31:45.293 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:45.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:44 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/4265636228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:31:45.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.899+0000 7f057c000640 1 -- 172.21.15.159:0/1657693788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0574109120 msgr2=0x7f0574110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:45.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.899+0000 7f057c000640 1 --2- 172.21.15.159:0/1657693788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0574109120 0x7f0574110ca0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f056c009ca0 tx=0x7f056c02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:31:45.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.899+0000 7f057c000640 1 -- 172.21.15.159:0/1657693788 shutdown_connections 2024-09-19T07:31:45.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.899+0000 7f057c000640 1 --2- 172.21.15.159:0/1657693788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0574109120 0x7f0574110ca0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:45.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.899+0000 7f057c000640 1 --2- 172.21.15.159:0/1657693788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05741049a0 0x7f05741089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:45.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.899+0000 7f057c000640 1 --2- 172.21.15.159:0/1657693788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0574103ff0 0x7f05741043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:45.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.899+0000 7f057c000640 1 -- 172.21.15.159:0/1657693788 >> 172.21.15.159:0/1657693788 conn(0x7f05740ffa90 msgr2=0x7f0574101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:45.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.899+0000 7f057c000640 1 -- 172.21.15.159:0/1657693788 shutdown_connections 2024-09-19T07:31:45.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.899+0000 7f057c000640 1 -- 172.21.15.159:0/1657693788 wait complete. 2024-09-19T07:31:45.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.900+0000 7f057c000640 1 Processor -- start 2024-09-19T07:31:45.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.900+0000 7f057c000640 1 -- start start 2024-09-19T07:31:45.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.900+0000 7f057c000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0574103ff0 0x7f057419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:45.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.900+0000 7f057c000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05741049a0 0x7f057419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:45.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.900+0000 7f057c000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0574109120 0x7f057419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:45.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.900+0000 7f057c000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0574114180 con 0x7f0574103ff0 2024-09-19T07:31:45.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.900+0000 7f057c000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0574114000 con 0x7f0574109120 2024-09-19T07:31:45.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.900+0000 7f057c000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0574114300 con 0x7f05741049a0 2024-09-19T07:31:45.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.901+0000 7f0579600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0574103ff0 0x7f057419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:45.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.901+0000 7f0579600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0574103ff0 0x7f057419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51158/0 (socket says 172.21.15.159:51158) 2024-09-19T07:31:45.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.901+0000 7f0579600640 1 -- 172.21.15.159:0/1011006444 learned_addr learned my addr 172.21.15.159:0/1011006444 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:45.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.901+0000 7f0578c00640 1 --2- 172.21.15.159:0/1011006444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05741049a0 0x7f057419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:45.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.901+0000 7f057a000640 1 --2- 172.21.15.159:0/1011006444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0574109120 0x7f057419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:45.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.901+0000 7f0579600640 1 -- 172.21.15.159:0/1011006444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05741049a0 msgr2=0x7f057419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:45.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.901+0000 7f0579600640 1 --2- 172.21.15.159:0/1011006444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05741049a0 0x7f057419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:45.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.901+0000 7f0579600640 1 -- 172.21.15.159:0/1011006444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0574109120 msgr2=0x7f057419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:45.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.901+0000 7f0579600640 1 --2- 172.21.15.159:0/1011006444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0574109120 0x7f057419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:45.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.901+0000 7f0579600640 1 -- 172.21.15.159:0/1011006444 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057419f5d0 con 0x7f0574103ff0 2024-09-19T07:31:45.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.901+0000 7f0579600640 1 --2- 172.21.15.159:0/1011006444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0574103ff0 0x7f057419a620 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f056800e9f0 tx=0x7f056800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:45.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.902+0000 7f0572000640 1 -- 172.21.15.159:0/1011006444 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056800cd80 con 0x7f0574103ff0 2024-09-19T07:31:45.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.902+0000 7f0572000640 1 -- 172.21.15.159:0/1011006444 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0568010910 con 0x7f0574103ff0 2024-09-19T07:31:45.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.902+0000 7f0572000640 1 -- 172.21.15.159:0/1011006444 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568018990 con 0x7f0574103ff0 2024-09-19T07:31:45.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.902+0000 7f057c000640 1 -- 172.21.15.159:0/1011006444 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057419f8c0 con 0x7f0574103ff0 2024-09-19T07:31:45.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.902+0000 7f057c000640 1 -- 172.21.15.159:0/1011006444 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05741a71a0 con 0x7f0574103ff0 2024-09-19T07:31:45.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.903+0000 7f057c000640 1 -- 172.21.15.159:0/1011006444 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05741043d0 con 0x7f0574103ff0 2024-09-19T07:31:45.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.905+0000 7f0572000640 1 -- 172.21.15.159:0/1011006444 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f0568010ac0 con 0x7f0574103ff0 2024-09-19T07:31:45.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.906+0000 7f0572000640 1 --2- 172.21.15.159:0/1011006444 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f055c0a3240 0x7f055c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:45.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.906+0000 7f0572000640 1 -- 172.21.15.159:0/1011006444 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f05680d03c0 con 0x7f0574103ff0 2024-09-19T07:31:45.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.907+0000 7f0578c00640 1 --2- 172.21.15.159:0/1011006444 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f055c0a3240 0x7f055c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:45.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.907+0000 7f0578c00640 1 --2- 172.21.15.159:0/1011006444 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f055c0a3240 0x7f055c0a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f057410aaf0 tx=0x7f0560005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:45.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:45.908+0000 7f0572000640 1 -- 172.21.15.159:0/1011006444 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f056809c730 con 0x7f0574103ff0 2024-09-19T07:31:46.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:46.120+0000 7f057c000640 1 -- 172.21.15.159:0/1011006444 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f05741a7370 con 0x7f0574103ff0 2024-09-19T07:31:46.120 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:45 smithi159 ceph-mon[30472]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T07:31:46.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:45 smithi203 ceph-mon[35335]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T07:31:46.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:45 smithi189 ceph-mon[34851]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T07:31:46.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:46.858+0000 7f0572000640 1 -- 172.21.15.159:0/1011006444 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f05680a0a30 con 0x7f0574103ff0 2024-09-19T07:31:46.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:46.861+0000 7f057c000640 1 -- 172.21.15.159:0/1011006444 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f055c0a3240 msgr2=0x7f055c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:46.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:46.861+0000 7f057c000640 1 --2- 172.21.15.159:0/1011006444 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f055c0a3240 0x7f055c0a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f057410aaf0 tx=0x7f0560005f90 comp rx=0 tx=0).stop 2024-09-19T07:31:46.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:46.861+0000 7f057c000640 1 -- 172.21.15.159:0/1011006444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0574103ff0 msgr2=0x7f057419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:46.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:46.861+0000 7f057c000640 1 --2- 172.21.15.159:0/1011006444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0574103ff0 0x7f057419a620 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f056800e9f0 tx=0x7f056800eeb0 comp rx=0 tx=0).stop 2024-09-19T07:31:46.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:46.862+0000 7f057c000640 1 -- 172.21.15.159:0/1011006444 shutdown_connections 2024-09-19T07:31:46.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:46.862+0000 7f057c000640 1 --2- 172.21.15.159:0/1011006444 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f055c0a3240 0x7f055c0a5700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:46.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:46.862+0000 7f057c000640 1 --2- 172.21.15.159:0/1011006444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0574109120 0x7f057419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:46.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:46.862+0000 7f057c000640 1 --2- 172.21.15.159:0/1011006444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05741049a0 0x7f057419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:46.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:46.862+0000 7f057c000640 1 --2- 172.21.15.159:0/1011006444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0574103ff0 0x7f057419a620 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:46.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:46.862+0000 7f057c000640 1 -- 172.21.15.159:0/1011006444 >> 172.21.15.159:0/1011006444 conn(0x7f05740ffa90 msgr2=0x7f0574100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:46.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:46.862+0000 7f057c000640 1 -- 172.21.15.159:0/1011006444 shutdown_connections 2024-09-19T07:31:46.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:46.862+0000 7f057c000640 1 -- 172.21.15.159:0/1011006444 wait complete. 2024-09-19T07:31:47.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:46 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1011006444' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T07:31:47.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:46 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1011006444' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T07:31:47.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:46 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1011006444' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T07:31:47.671 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-19T07:31:47.671 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-19T07:31:47.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:47 smithi159 ceph-mon[30472]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T07:31:47.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:47 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1011006444' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T07:31:47.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:47 smithi159 ceph-mon[30472]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:31:48.044 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:48.187 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:47 smithi203 ceph-mon[35335]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T07:31:48.188 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:47 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1011006444' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T07:31:48.188 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:47 smithi203 ceph-mon[35335]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:31:48.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:47 smithi189 ceph-mon[34851]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-19T07:31:48.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:47 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1011006444' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T07:31:48.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:47 smithi189 ceph-mon[34851]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:31:48.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.676+0000 7fec39200640 1 -- 172.21.15.159:0/1386845619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec34101de0 msgr2=0x7fec341021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:48.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.676+0000 7fec39200640 1 --2- 172.21.15.159:0/1386845619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec34101de0 0x7fec341021c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fec24009900 tx=0x7fec2402a410 comp rx=0 tx=0).stop 2024-09-19T07:31:48.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.676+0000 7fec39200640 1 -- 172.21.15.159:0/1386845619 shutdown_connections 2024-09-19T07:31:48.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.676+0000 7fec39200640 1 --2- 172.21.15.159:0/1386845619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec34106f10 0x7fec3410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:48.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.676+0000 7fec39200640 1 --2- 172.21.15.159:0/1386845619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec34102790 0x7fec341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:48.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.676+0000 7fec39200640 1 --2- 172.21.15.159:0/1386845619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec34101de0 0x7fec341021c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:48.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.676+0000 7fec39200640 1 -- 172.21.15.159:0/1386845619 >> 172.21.15.159:0/1386845619 conn(0x7fec340fd8c0 msgr2=0x7fec340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:48.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.676+0000 7fec39200640 1 -- 172.21.15.159:0/1386845619 shutdown_connections 2024-09-19T07:31:48.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.676+0000 7fec39200640 1 -- 172.21.15.159:0/1386845619 wait complete. 2024-09-19T07:31:48.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.678+0000 7fec39200640 1 Processor -- start 2024-09-19T07:31:48.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.678+0000 7fec39200640 1 -- start start 2024-09-19T07:31:48.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.678+0000 7fec39200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec34101de0 0x7fec341983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:48.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.678+0000 7fec39200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec34102790 0x7fec34198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:48.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.678+0000 7fec39200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec34106f10 0x7fec3419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:48.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.678+0000 7fec39200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fec34111f70 con 0x7fec34102790 2024-09-19T07:31:48.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.678+0000 7fec39200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec34111df0 con 0x7fec34101de0 2024-09-19T07:31:48.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.678+0000 7fec39200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fec341120f0 con 0x7fec34106f10 2024-09-19T07:31:48.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.679+0000 7fec31400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec34102790 0x7fec34198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:48.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.679+0000 7fec31e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec34101de0 0x7fec341983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:48.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.679+0000 7fec31400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec34102790 0x7fec34198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51180/0 (socket says 172.21.15.159:51180) 2024-09-19T07:31:48.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.679+0000 7fec31400640 1 -- 172.21.15.159:0/431194934 learned_addr learned my addr 172.21.15.159:0/431194934 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:48.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.679+0000 7fec32800640 1 --2- 172.21.15.159:0/431194934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec34106f10 0x7fec3419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:48.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.679+0000 7fec31400640 1 -- 172.21.15.159:0/431194934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec34106f10 msgr2=0x7fec3419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:48.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.679+0000 7fec31400640 1 --2- 172.21.15.159:0/431194934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec34106f10 0x7fec3419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:48.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.679+0000 7fec31400640 1 -- 172.21.15.159:0/431194934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec34101de0 msgr2=0x7fec341983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:48.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.679+0000 7fec31400640 1 --2- 172.21.15.159:0/431194934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec34101de0 0x7fec341983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:48.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.679+0000 7fec31400640 1 -- 172.21.15.159:0/431194934 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3419d3a0 con 0x7fec34102790 2024-09-19T07:31:48.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.680+0000 7fec31400640 1 --2- 172.21.15.159:0/431194934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec34102790 0x7fec34198930 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fec2000d950 tx=0x7fec2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:48.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.680+0000 7fec2aa00640 1 -- 172.21.15.159:0/431194934 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20017070 con 0x7fec34102790 2024-09-19T07:31:48.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.680+0000 7fec2aa00640 1 -- 172.21.15.159:0/431194934 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec20009c30 con 0x7fec34102790 2024-09-19T07:31:48.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.680+0000 7fec39200640 1 -- 172.21.15.159:0/431194934 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3419d690 con 0x7fec34102790 2024-09-19T07:31:48.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.680+0000 7fec2aa00640 1 -- 172.21.15.159:0/431194934 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2000f6b0 con 0x7fec34102790 2024-09-19T07:31:48.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.680+0000 7fec39200640 1 -- 172.21.15.159:0/431194934 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec341a4f70 con 0x7fec34102790 2024-09-19T07:31:48.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.681+0000 7fec39200640 1 -- 172.21.15.159:0/431194934 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec341021c0 con 0x7fec34102790 2024-09-19T07:31:48.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.683+0000 7fec2aa00640 1 -- 172.21.15.159:0/431194934 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fec200109d0 con 0x7fec34102790 2024-09-19T07:31:48.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.684+0000 7fec2aa00640 1 --2- 172.21.15.159:0/431194934 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fec180a3240 0x7fec180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:48.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.684+0000 7fec2aa00640 1 -- 172.21.15.159:0/431194934 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fec200cf890 con 0x7fec34102790 2024-09-19T07:31:48.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.684+0000 7fec31e00640 1 --2- 172.21.15.159:0/431194934 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fec180a3240 0x7fec180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:48.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.685+0000 7fec31e00640 1 --2- 172.21.15.159:0/431194934 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fec180a3240 0x7fec180a5700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fec24009230 tx=0x7fec240058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:48.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.688+0000 7fec2aa00640 1 -- 172.21.15.159:0/431194934 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fec20009de0 con 0x7fec34102790 2024-09-19T07:31:48.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:48.902+0000 7fec39200640 1 -- 172.21.15.159:0/431194934 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fec3410a150 con 0x7fec34102790 2024-09-19T07:31:49.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:49.867+0000 7fec2aa00640 1 -- 172.21.15.159:0/431194934 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fec2009bba0 con 0x7fec34102790 2024-09-19T07:31:49.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:49.870+0000 7fec39200640 1 -- 172.21.15.159:0/431194934 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fec180a3240 msgr2=0x7fec180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:49.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:49.870+0000 7fec39200640 1 --2- 172.21.15.159:0/431194934 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fec180a3240 0x7fec180a5700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fec24009230 tx=0x7fec240058d0 comp rx=0 tx=0).stop 2024-09-19T07:31:49.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:49.870+0000 7fec39200640 1 -- 172.21.15.159:0/431194934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec34102790 msgr2=0x7fec34198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:49.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:49.870+0000 7fec39200640 1 --2- 172.21.15.159:0/431194934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec34102790 0x7fec34198930 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fec2000d950 tx=0x7fec2000de10 comp rx=0 tx=0).stop 2024-09-19T07:31:49.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:49.870+0000 7fec39200640 1 -- 172.21.15.159:0/431194934 shutdown_connections 2024-09-19T07:31:49.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:49.870+0000 7fec39200640 1 --2- 172.21.15.159:0/431194934 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fec180a3240 0x7fec180a5700 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:49.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:49.870+0000 7fec39200640 1 --2- 172.21.15.159:0/431194934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec34106f10 0x7fec3419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:49.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:49.870+0000 7fec39200640 1 --2- 172.21.15.159:0/431194934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec34102790 0x7fec34198930 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:49.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:49.870+0000 7fec39200640 1 --2- 172.21.15.159:0/431194934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec34101de0 0x7fec341983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:49.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:49.870+0000 7fec39200640 1 -- 172.21.15.159:0/431194934 >> 172.21.15.159:0/431194934 conn(0x7fec340fd8c0 msgr2=0x7fec340fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:49.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:49.870+0000 7fec39200640 1 -- 172.21.15.159:0/431194934 shutdown_connections 2024-09-19T07:31:49.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:49.870+0000 7fec39200640 1 -- 172.21.15.159:0/431194934 wait complete. 2024-09-19T07:31:50.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:49 smithi159 ceph-mon[30472]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T07:31:50.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:49 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/431194934' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-19T07:31:50.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:49 smithi203 ceph-mon[35335]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T07:31:50.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:49 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/431194934' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-19T07:31:50.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:49 smithi189 ceph-mon[34851]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T07:31:50.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:49 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/431194934' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-19T07:31:50.677 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-19T07:31:50.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:50 smithi159 ceph-mon[30472]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T07:31:50.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:50 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/431194934' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-19T07:31:50.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:50 smithi159 ceph-mon[30472]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T07:31:50.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:50 smithi159 ceph-mon[30472]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:31:50.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:50 smithi159 ceph-mon[30472]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T07:31:50.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:50 smithi159 ceph-mon[30472]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T07:31:51.058 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:51.289 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:50 smithi203 ceph-mon[35335]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T07:31:51.289 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:50 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/431194934' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-19T07:31:51.289 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:50 smithi203 ceph-mon[35335]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T07:31:51.289 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:50 smithi203 ceph-mon[35335]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:31:51.290 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:50 smithi203 ceph-mon[35335]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T07:31:51.290 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:50 smithi203 ceph-mon[35335]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T07:31:51.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:50 smithi189 ceph-mon[34851]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T07:31:51.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:50 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/431194934' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-19T07:31:51.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:50 smithi189 ceph-mon[34851]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T07:31:51.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:50 smithi189 ceph-mon[34851]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:31:51.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:50 smithi189 ceph-mon[34851]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T07:31:51.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:50 smithi189 ceph-mon[34851]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T07:31:51.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.637+0000 7fa36ca00640 1 -- 172.21.15.159:0/2774130855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa368101df0 msgr2=0x7fa3681021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:51.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.637+0000 7fa36ca00640 1 --2- 172.21.15.159:0/2774130855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa368101df0 0x7fa3681021d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa358009900 tx=0x7fa35802a410 comp rx=0 tx=0).stop 2024-09-19T07:31:51.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.637+0000 7fa36ca00640 1 -- 172.21.15.159:0/2774130855 shutdown_connections 2024-09-19T07:31:51.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.637+0000 7fa36ca00640 1 --2- 172.21.15.159:0/2774130855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa368106f20 0x7fa36810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:51.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.637+0000 7fa36ca00640 1 --2- 172.21.15.159:0/2774130855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3681027a0 0x7fa3681067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:51.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.637+0000 7fa36ca00640 1 --2- 172.21.15.159:0/2774130855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa368101df0 0x7fa3681021d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:51.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.637+0000 7fa36ca00640 1 -- 172.21.15.159:0/2774130855 >> 172.21.15.159:0/2774130855 conn(0x7fa3680fd8b0 msgr2=0x7fa3680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:51.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.638+0000 7fa36ca00640 1 -- 172.21.15.159:0/2774130855 shutdown_connections 2024-09-19T07:31:51.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.638+0000 7fa36ca00640 1 -- 172.21.15.159:0/2774130855 wait complete. 2024-09-19T07:31:51.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.639+0000 7fa36ca00640 1 Processor -- start 2024-09-19T07:31:51.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.639+0000 7fa36ca00640 1 -- start start 2024-09-19T07:31:51.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.639+0000 7fa36ca00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa368101df0 0x7fa36819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:51.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.639+0000 7fa36ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3681027a0 0x7fa36819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:51.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.639+0000 7fa36ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa368106f20 0x7fa36819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:51.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.639+0000 7fa36ca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa368111f80 con 0x7fa3681027a0 2024-09-19T07:31:51.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.639+0000 7fa36ca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa368111e00 con 0x7fa368106f20 2024-09-19T07:31:51.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.639+0000 7fa36ca00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa368112100 con 0x7fa368101df0 2024-09-19T07:31:51.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.640+0000 7fa35fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3681027a0 0x7fa36819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:51.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.640+0000 7fa365e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa368101df0 0x7fa36819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:51.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.640+0000 7fa366800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa368106f20 0x7fa36819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:51.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.640+0000 7fa365e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa368101df0 0x7fa36819a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:38200/0 (socket says 172.21.15.159:38200) 2024-09-19T07:31:51.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.640+0000 7fa366800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa368106f20 0x7fa36819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:41294/0 (socket says 172.21.15.159:41294) 2024-09-19T07:31:51.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.640+0000 7fa365e00640 1 -- 172.21.15.159:0/2381212174 learned_addr learned my addr 172.21.15.159:0/2381212174 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:51.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.640+0000 7fa366800640 1 -- 172.21.15.159:0/2381212174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa368101df0 msgr2=0x7fa36819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:51.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.640+0000 7fa366800640 1 --2- 172.21.15.159:0/2381212174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa368101df0 0x7fa36819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:51.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.640+0000 7fa366800640 1 -- 172.21.15.159:0/2381212174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3681027a0 msgr2=0x7fa36819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:51.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.640+0000 7fa366800640 1 --2- 172.21.15.159:0/2381212174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3681027a0 0x7fa36819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:51.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.640+0000 7fa366800640 1 -- 172.21.15.159:0/2381212174 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36819f590 con 0x7fa368106f20 2024-09-19T07:31:51.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.640+0000 7fa35fe00640 1 --2- 172.21.15.159:0/2381212174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3681027a0 0x7fa36819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:31:51.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.640+0000 7fa366800640 1 --2- 172.21.15.159:0/2381212174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa368106f20 0x7fa36819eeb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa360009fd0 tx=0x7fa36000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:51.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.641+0000 7fa35ea00640 1 -- 172.21.15.159:0/2381212174 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36000ce20 con 0x7fa368106f20 2024-09-19T07:31:51.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.641+0000 7fa36ca00640 1 -- 172.21.15.159:0/2381212174 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36819f880 con 0x7fa368106f20 2024-09-19T07:31:51.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.641+0000 7fa35ea00640 1 -- 172.21.15.159:0/2381212174 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa360005020 con 0x7fa368106f20 2024-09-19T07:31:51.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.641+0000 7fa35ea00640 1 -- 172.21.15.159:0/2381212174 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360010720 con 0x7fa368106f20 2024-09-19T07:31:51.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.641+0000 7fa36ca00640 1 -- 172.21.15.159:0/2381212174 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3681a7160 con 0x7fa368106f20 2024-09-19T07:31:51.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.642+0000 7fa36ca00640 1 -- 172.21.15.159:0/2381212174 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3681021d0 con 0x7fa368106f20 2024-09-19T07:31:51.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.644+0000 7fa35ea00640 1 -- 172.21.15.159:0/2381212174 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa360005540 con 0x7fa368106f20 2024-09-19T07:31:51.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.645+0000 7fa35ea00640 1 --2- 172.21.15.159:0/2381212174 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa34c0a3240 0x7fa34c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:51.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.645+0000 7fa35ea00640 1 -- 172.21.15.159:0/2381212174 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fa3600d03e0 con 0x7fa368106f20 2024-09-19T07:31:51.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.646+0000 7fa365e00640 1 --2- 172.21.15.159:0/2381212174 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa34c0a3240 0x7fa34c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:51.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.646+0000 7fa365e00640 1 --2- 172.21.15.159:0/2381212174 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa34c0a3240 0x7fa34c0a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa3580096c0 tx=0x7fa358005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:51.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.647+0000 7fa35ea00640 1 -- 172.21.15.159:0/2381212174 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa36009cdd0 con 0x7fa368106f20 2024-09-19T07:31:51.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.868+0000 7fa36ca00640 1 -- 172.21.15.159:0/2381212174 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fa36819b9e0 con 0x7fa368106f20 2024-09-19T07:31:51.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.889+0000 7fa35ea00640 1 -- 172.21.15.159:0/2381212174 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7fa36819b9e0 con 0x7fa368106f20 2024-09-19T07:31:51.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.892+0000 7fa36ca00640 1 -- 172.21.15.159:0/2381212174 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa34c0a3240 msgr2=0x7fa34c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:51.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.892+0000 7fa36ca00640 1 --2- 172.21.15.159:0/2381212174 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa34c0a3240 0x7fa34c0a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa3580096c0 tx=0x7fa358005a20 comp rx=0 tx=0).stop 2024-09-19T07:31:51.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.892+0000 7fa36ca00640 1 -- 172.21.15.159:0/2381212174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa368106f20 msgr2=0x7fa36819eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:51.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.892+0000 7fa36ca00640 1 --2- 172.21.15.159:0/2381212174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa368106f20 0x7fa36819eeb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa360009fd0 tx=0x7fa36000ef10 comp rx=0 tx=0).stop 2024-09-19T07:31:51.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.892+0000 7fa36ca00640 1 -- 172.21.15.159:0/2381212174 shutdown_connections 2024-09-19T07:31:51.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.892+0000 7fa36ca00640 1 --2- 172.21.15.159:0/2381212174 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa34c0a3240 0x7fa34c0a5700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:51.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.892+0000 7fa36ca00640 1 --2- 172.21.15.159:0/2381212174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa368106f20 0x7fa36819eeb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:51.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.892+0000 7fa36ca00640 1 --2- 172.21.15.159:0/2381212174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3681027a0 0x7fa36819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:51.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.892+0000 7fa36ca00640 1 --2- 172.21.15.159:0/2381212174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa368101df0 0x7fa36819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:51.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.892+0000 7fa36ca00640 1 -- 172.21.15.159:0/2381212174 >> 172.21.15.159:0/2381212174 conn(0x7fa3680fd8b0 msgr2=0x7fa3680ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:51.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.892+0000 7fa36ca00640 1 -- 172.21.15.159:0/2381212174 shutdown_connections 2024-09-19T07:31:51.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:51.892+0000 7fa36ca00640 1 -- 172.21.15.159:0/2381212174 wait complete. 2024-09-19T07:31:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:51 smithi159 ceph-mon[30472]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T07:31:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:51 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] up:active 2024-09-19T07:31:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:51 smithi159 ceph-mon[30472]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T07:31:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:51 smithi159 ceph-mon[30472]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T07:31:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:51 smithi159 ceph-mon[30472]: Cluster is now healthy 2024-09-19T07:31:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:51 smithi159 ceph-mon[30472]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T07:31:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:51 smithi159 ceph-mon[30472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-19T07:31:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:51 smithi159 ceph-mon[30472]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-19T07:31:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:51 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2381212174' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T07:31:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:51 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T07:31:52.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:51 smithi203 ceph-mon[35335]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T07:31:52.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:51 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] up:active 2024-09-19T07:31:52.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:51 smithi203 ceph-mon[35335]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T07:31:52.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:51 smithi203 ceph-mon[35335]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T07:31:52.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:51 smithi203 ceph-mon[35335]: Cluster is now healthy 2024-09-19T07:31:52.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:51 smithi203 ceph-mon[35335]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T07:31:52.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:51 smithi203 ceph-mon[35335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-19T07:31:52.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:51 smithi203 ceph-mon[35335]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-19T07:31:52.288 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:51 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/2381212174' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T07:31:52.288 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:51 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T07:31:52.309 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:51 smithi189 ceph-mon[34851]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-19T07:31:52.309 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:51 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] up:active 2024-09-19T07:31:52.309 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:51 smithi189 ceph-mon[34851]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T07:31:52.309 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:51 smithi189 ceph-mon[34851]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T07:31:52.310 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:51 smithi189 ceph-mon[34851]: Cluster is now healthy 2024-09-19T07:31:52.310 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:51 smithi189 ceph-mon[34851]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T07:31:52.310 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:51 smithi189 ceph-mon[34851]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-19T07:31:52.310 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:51 smithi189 ceph-mon[34851]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-19T07:31:52.310 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:51 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/2381212174' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T07:31:52.310 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:51 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T07:31:52.749 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-19T07:31:53.046 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:52 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T07:31:53.046 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:52 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] up:active 2024-09-19T07:31:53.046 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:52 smithi159 ceph-mon[30472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-19T07:31:53.046 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:52 smithi159 ceph-mon[30472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T07:31:53.132 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:53.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:52 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T07:31:53.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:52 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] up:active 2024-09-19T07:31:53.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:52 smithi203 ceph-mon[35335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-19T07:31:53.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:52 smithi203 ceph-mon[35335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T07:31:53.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:52 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T07:31:53.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:52 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] up:active 2024-09-19T07:31:53.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:52 smithi189 ceph-mon[34851]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-19T07:31:53.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:52 smithi189 ceph-mon[34851]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T07:31:53.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.728+0000 7f6a10a00640 1 -- 172.21.15.159:0/780511984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a0c077040 msgr2=0x7f6a0c112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:53.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.728+0000 7f6a10a00640 1 --2- 172.21.15.159:0/780511984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a0c077040 0x7f6a0c112330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009900 tx=0x7f69f802a410 comp rx=0 tx=0).stop 2024-09-19T07:31:53.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.728+0000 7f6a10a00640 1 -- 172.21.15.159:0/780511984 shutdown_connections 2024-09-19T07:31:53.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.728+0000 7f6a10a00640 1 --2- 172.21.15.159:0/780511984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a0c112870 0x7f6a0c114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:53.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.728+0000 7f6a10a00640 1 --2- 172.21.15.159:0/780511984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a0c077040 0x7f6a0c112330 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:53.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.728+0000 7f6a10a00640 1 --2- 172.21.15.159:0/780511984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a0c076720 0x7f6a0c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:53.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.728+0000 7f6a10a00640 1 -- 172.21.15.159:0/780511984 >> 172.21.15.159:0/780511984 conn(0x7f6a0c0ff8d0 msgr2=0x7f6a0c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:53.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.728+0000 7f6a10a00640 1 -- 172.21.15.159:0/780511984 shutdown_connections 2024-09-19T07:31:53.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.728+0000 7f6a10a00640 1 -- 172.21.15.159:0/780511984 wait complete. 2024-09-19T07:31:53.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.729+0000 7f6a10a00640 1 Processor -- start 2024-09-19T07:31:53.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.729+0000 7f6a10a00640 1 -- start start 2024-09-19T07:31:53.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.730+0000 7f6a10a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a0c076720 0x7f6a0c1a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:53.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.730+0000 7f6a10a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a0c077040 0x7f6a0c1a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:53.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.730+0000 7f6a10a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a0c112870 0x7f6a0c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:53.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.730+0000 7f6a10a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c118450 con 0x7f6a0c112870 2024-09-19T07:31:53.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.730+0000 7f6a10a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1182d0 con 0x7f6a0c076720 2024-09-19T07:31:53.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.730+0000 7f6a10a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1185d0 con 0x7f6a0c077040 2024-09-19T07:31:53.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.730+0000 7f6a0be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a0c112870 0x7f6a0c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:53.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.730+0000 7f6a0aa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a0c077040 0x7f6a0c1a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:53.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.730+0000 7f6a0be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a0c112870 0x7f6a0c19e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44936/0 (socket says 172.21.15.159:44936) 2024-09-19T07:31:53.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.730+0000 7f6a0be00640 1 -- 172.21.15.159:0/2988866244 learned_addr learned my addr 172.21.15.159:0/2988866244 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:53.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.730+0000 7f6a0b400640 1 --2- 172.21.15.159:0/2988866244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a0c076720 0x7f6a0c1a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:53.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.730+0000 7f6a0aa00640 1 -- 172.21.15.159:0/2988866244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a0c076720 msgr2=0x7f6a0c1a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:53.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.730+0000 7f6a0aa00640 1 --2- 172.21.15.159:0/2988866244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a0c076720 0x7f6a0c1a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:53.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.730+0000 7f6a0aa00640 1 -- 172.21.15.159:0/2988866244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a0c112870 msgr2=0x7f6a0c19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:53.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.730+0000 7f6a0aa00640 1 --2- 172.21.15.159:0/2988866244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a0c112870 0x7f6a0c19e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:53.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.731+0000 7f6a0aa00640 1 -- 172.21.15.159:0/2988866244 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c19efa0 con 0x7f6a0c077040 2024-09-19T07:31:53.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.731+0000 7f6a0aa00640 1 --2- 172.21.15.159:0/2988866244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a0c077040 0x7f6a0c1a4ba0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f69f802a930 tx=0x7f69f8003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:53.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.731+0000 7f6a03e00640 1 -- 172.21.15.159:0/2988866244 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f803f070 con 0x7f6a0c077040 2024-09-19T07:31:53.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.731+0000 7f6a10a00640 1 -- 172.21.15.159:0/2988866244 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c19f230 con 0x7f6a0c077040 2024-09-19T07:31:53.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.731+0000 7f6a03e00640 1 -- 172.21.15.159:0/2988866244 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69f8031c90 con 0x7f6a0c077040 2024-09-19T07:31:53.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.731+0000 7f6a03e00640 1 -- 172.21.15.159:0/2988866244 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8035030 con 0x7f6a0c077040 2024-09-19T07:31:53.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.731+0000 7f6a10a00640 1 -- 172.21.15.159:0/2988866244 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c1ab410 con 0x7f6a0c077040 2024-09-19T07:31:53.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.732+0000 7f6a10a00640 1 -- 172.21.15.159:0/2988866244 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0c077e60 con 0x7f6a0c077040 2024-09-19T07:31:53.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.735+0000 7f6a03e00640 1 -- 172.21.15.159:0/2988866244 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f69f8042070 con 0x7f6a0c077040 2024-09-19T07:31:53.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.736+0000 7f6a03e00640 1 --2- 172.21.15.159:0/2988866244 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f69f00a3240 0x7f69f00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:53.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.736+0000 7f6a03e00640 1 -- 172.21.15.159:0/2988866244 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f69f80f0c70 con 0x7f6a0c077040 2024-09-19T07:31:53.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.737+0000 7f6a0b400640 1 --2- 172.21.15.159:0/2988866244 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f69f00a3240 0x7f69f00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:53.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.737+0000 7f6a0b400640 1 --2- 172.21.15.159:0/2988866244 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f69f00a3240 0x7f69f00a5700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009bd0 tx=0x7f69fc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:53.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.738+0000 7f6a03e00640 1 -- 172.21.15.159:0/2988866244 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f69f80bcf80 con 0x7f6a0c077040 2024-09-19T07:31:53.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:53.952+0000 7f6a10a00640 1 -- 172.21.15.159:0/2988866244 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f6a0c10da00 con 0x7f6a0c077040 2024-09-19T07:31:54.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:53 smithi159 ceph-mon[30472]: pgmap v339: 129 pgs: 129 active+clean; 583 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-19T07:31:54.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:53 smithi203 ceph-mon[35335]: pgmap v339: 129 pgs: 129 active+clean; 583 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-19T07:31:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:53 smithi189 ceph-mon[34851]: pgmap v339: 129 pgs: 129 active+clean; 583 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-19T07:31:54.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:54.901+0000 7f6a03e00640 1 -- 172.21.15.159:0/2988866244 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f69f80c1280 con 0x7f6a0c077040 2024-09-19T07:31:54.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:54.904+0000 7f6a10a00640 1 -- 172.21.15.159:0/2988866244 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f69f00a3240 msgr2=0x7f69f00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:54.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:54.904+0000 7f6a10a00640 1 --2- 172.21.15.159:0/2988866244 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f69f00a3240 0x7f69f00a5700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009bd0 tx=0x7f69fc005f90 comp rx=0 tx=0).stop 2024-09-19T07:31:54.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:54.904+0000 7f6a10a00640 1 -- 172.21.15.159:0/2988866244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a0c077040 msgr2=0x7f6a0c1a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:54.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:54.904+0000 7f6a10a00640 1 --2- 172.21.15.159:0/2988866244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a0c077040 0x7f6a0c1a4ba0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f69f802a930 tx=0x7f69f8003b20 comp rx=0 tx=0).stop 2024-09-19T07:31:54.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:54.904+0000 7f6a10a00640 1 -- 172.21.15.159:0/2988866244 shutdown_connections 2024-09-19T07:31:54.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:54.904+0000 7f6a10a00640 1 --2- 172.21.15.159:0/2988866244 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f69f00a3240 0x7f69f00a5700 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:54.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:54.904+0000 7f6a10a00640 1 --2- 172.21.15.159:0/2988866244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a0c112870 0x7f6a0c19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:54.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:54.904+0000 7f6a10a00640 1 --2- 172.21.15.159:0/2988866244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a0c077040 0x7f6a0c1a4ba0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:54.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:54.904+0000 7f6a10a00640 1 --2- 172.21.15.159:0/2988866244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a0c076720 0x7f6a0c1a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:54.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:54.904+0000 7f6a10a00640 1 -- 172.21.15.159:0/2988866244 >> 172.21.15.159:0/2988866244 conn(0x7f6a0c0ff8d0 msgr2=0x7f6a0c101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:54.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:54.904+0000 7f6a10a00640 1 -- 172.21.15.159:0/2988866244 shutdown_connections 2024-09-19T07:31:54.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:54.904+0000 7f6a10a00640 1 -- 172.21.15.159:0/2988866244 wait complete. 2024-09-19T07:31:55.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:54 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/2988866244' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T07:31:55.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:54 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T07:31:55.056 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:31:54 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-b[64691]: 2024-09-19T07:31:54.744+0000 7f8c4eb2e640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-19T07:31:55.056 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:31:54 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-b[64691]: 2024-09-19T07:31:54.744+0000 7f8c4eb2e640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T07:31:55.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:54 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/2988866244' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T07:31:55.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:54 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T07:31:55.293 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:54 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2988866244' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T07:31:55.294 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:54 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T07:31:55.744 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-19T07:31:55.745 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-19T07:31:56.120 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:56.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:55 smithi159 ceph-mon[30472]: pgmap v340: 129 pgs: 129 active+clean; 584 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-19T07:31:56.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:55 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T07:31:56.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:55 smithi159 ceph-mon[30472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T07:31:56.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:55 smithi203 ceph-mon[35335]: pgmap v340: 129 pgs: 129 active+clean; 584 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-19T07:31:56.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:55 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T07:31:56.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:55 smithi203 ceph-mon[35335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T07:31:56.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:55 smithi189 ceph-mon[34851]: pgmap v340: 129 pgs: 129 active+clean; 584 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-19T07:31:56.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:55 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T07:31:56.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:55 smithi189 ceph-mon[34851]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T07:31:56.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.710+0000 7fc019400640 1 -- 172.21.15.159:0/2754631528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc014103ff0 msgr2=0x7fc0141043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:56.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.710+0000 7fc019400640 1 --2- 172.21.15.159:0/2754631528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc014103ff0 0x7fc0141043d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc004009900 tx=0x7fc00402a3f0 comp rx=0 tx=0).stop 2024-09-19T07:31:56.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.710+0000 7fc019400640 1 -- 172.21.15.159:0/2754631528 shutdown_connections 2024-09-19T07:31:56.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.710+0000 7fc019400640 1 --2- 172.21.15.159:0/2754631528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc014109120 0x7fc014110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:56.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.710+0000 7fc019400640 1 --2- 172.21.15.159:0/2754631528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0141049a0 0x7fc0141089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:56.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.710+0000 7fc019400640 1 --2- 172.21.15.159:0/2754631528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc014103ff0 0x7fc0141043d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:56.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.710+0000 7fc019400640 1 -- 172.21.15.159:0/2754631528 >> 172.21.15.159:0/2754631528 conn(0x7fc0140ffa90 msgr2=0x7fc014101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:56.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.710+0000 7fc019400640 1 -- 172.21.15.159:0/2754631528 shutdown_connections 2024-09-19T07:31:56.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.710+0000 7fc019400640 1 -- 172.21.15.159:0/2754631528 wait complete. 2024-09-19T07:31:56.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.711+0000 7fc019400640 1 Processor -- start 2024-09-19T07:31:56.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.711+0000 7fc019400640 1 -- start start 2024-09-19T07:31:56.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.711+0000 7fc019400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc014103ff0 0x7fc01419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:56.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.711+0000 7fc019400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0141049a0 0x7fc01419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:56.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.712+0000 7fc019400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc014109120 0x7fc01419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:56.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.712+0000 7fc019400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc014114140 con 0x7fc014103ff0 2024-09-19T07:31:56.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.712+0000 7fc019400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc014113fc0 con 0x7fc0141049a0 2024-09-19T07:31:56.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.712+0000 7fc019400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141142c0 con 0x7fc014109120 2024-09-19T07:31:56.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.712+0000 7fc012800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc014103ff0 0x7fc01419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:56.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.712+0000 7fc012800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc014103ff0 0x7fc01419a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44948/0 (socket says 172.21.15.159:44948) 2024-09-19T07:31:56.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.712+0000 7fc012800640 1 -- 172.21.15.159:0/2621251621 learned_addr learned my addr 172.21.15.159:0/2621251621 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:56.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.712+0000 7fc011e00640 1 --2- 172.21.15.159:0/2621251621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0141049a0 0x7fc01419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:56.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.712+0000 7fc013200640 1 --2- 172.21.15.159:0/2621251621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc014109120 0x7fc01419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:56.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.712+0000 7fc012800640 1 -- 172.21.15.159:0/2621251621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc014109120 msgr2=0x7fc01419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:56.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.712+0000 7fc012800640 1 --2- 172.21.15.159:0/2621251621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc014109120 0x7fc01419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:56.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.712+0000 7fc012800640 1 -- 172.21.15.159:0/2621251621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0141049a0 msgr2=0x7fc01419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:56.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.712+0000 7fc012800640 1 --2- 172.21.15.159:0/2621251621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0141049a0 0x7fc01419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:56.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.712+0000 7fc012800640 1 -- 172.21.15.159:0/2621251621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01419f5b0 con 0x7fc014103ff0 2024-09-19T07:31:56.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.713+0000 7fc012800640 1 --2- 172.21.15.159:0/2621251621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc014103ff0 0x7fc01419a600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc004006010 tx=0x7fc0040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:56.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.713+0000 7fc00b400640 1 -- 172.21.15.159:0/2621251621 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00403f030 con 0x7fc014103ff0 2024-09-19T07:31:56.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.713+0000 7fc00b400640 1 -- 172.21.15.159:0/2621251621 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc004003ce0 con 0x7fc014103ff0 2024-09-19T07:31:56.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.713+0000 7fc00b400640 1 -- 172.21.15.159:0/2621251621 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004035070 con 0x7fc014103ff0 2024-09-19T07:31:56.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.713+0000 7fc019400640 1 -- 172.21.15.159:0/2621251621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01419f840 con 0x7fc014103ff0 2024-09-19T07:31:56.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.713+0000 7fc019400640 1 -- 172.21.15.159:0/2621251621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0141a7180 con 0x7fc014103ff0 2024-09-19T07:31:56.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.714+0000 7fc019400640 1 -- 172.21.15.159:0/2621251621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0141043d0 con 0x7fc014103ff0 2024-09-19T07:31:56.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.716+0000 7fc00b400640 1 -- 172.21.15.159:0/2621251621 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc004042030 con 0x7fc014103ff0 2024-09-19T07:31:56.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.717+0000 7fc00b400640 1 --2- 172.21.15.159:0/2621251621 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fbff80a32c0 0x7fbff80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:56.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.717+0000 7fc00b400640 1 -- 172.21.15.159:0/2621251621 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fc0040f03e0 con 0x7fc014103ff0 2024-09-19T07:31:56.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.717+0000 7fc011e00640 1 --2- 172.21.15.159:0/2621251621 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fbff80a32c0 0x7fbff80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:56.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.718+0000 7fc011e00640 1 --2- 172.21.15.159:0/2621251621 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fbff80a32c0 0x7fbff80a5780 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc0140fff20 tx=0x7fc000005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:56.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.719+0000 7fc00b400640 1 -- 172.21.15.159:0/2621251621 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc0040bce60 con 0x7fc014103ff0 2024-09-19T07:31:56.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.923+0000 7fc019400640 1 -- 172.21.15.159:0/2621251621 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc014062630 con 0x7fbff80a32c0 2024-09-19T07:31:56.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.954+0000 7fc00b400640 1 -- 172.21.15.159:0/2621251621 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc014062630 con 0x7fbff80a32c0 2024-09-19T07:31:56.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.957+0000 7fc019400640 1 -- 172.21.15.159:0/2621251621 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fbff80a32c0 msgr2=0x7fbff80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:56.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.957+0000 7fc019400640 1 --2- 172.21.15.159:0/2621251621 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fbff80a32c0 0x7fbff80a5780 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc0140fff20 tx=0x7fc000005f90 comp rx=0 tx=0).stop 2024-09-19T07:31:56.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.957+0000 7fc019400640 1 -- 172.21.15.159:0/2621251621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc014103ff0 msgr2=0x7fc01419a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:56.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.957+0000 7fc019400640 1 --2- 172.21.15.159:0/2621251621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc014103ff0 0x7fc01419a600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc004006010 tx=0x7fc0040042f0 comp rx=0 tx=0).stop 2024-09-19T07:31:56.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.957+0000 7fc019400640 1 -- 172.21.15.159:0/2621251621 shutdown_connections 2024-09-19T07:31:56.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.957+0000 7fc019400640 1 --2- 172.21.15.159:0/2621251621 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fbff80a32c0 0x7fbff80a5780 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:56.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.957+0000 7fc019400640 1 --2- 172.21.15.159:0/2621251621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc014109120 0x7fc01419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:56.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.957+0000 7fc019400640 1 --2- 172.21.15.159:0/2621251621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0141049a0 0x7fc01419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:56.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.957+0000 7fc019400640 1 --2- 172.21.15.159:0/2621251621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc014103ff0 0x7fc01419a600 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:56.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.957+0000 7fc019400640 1 -- 172.21.15.159:0/2621251621 >> 172.21.15.159:0/2621251621 conn(0x7fc0140ffa90 msgr2=0x7fc014100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:56.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.957+0000 7fc019400640 1 -- 172.21.15.159:0/2621251621 shutdown_connections 2024-09-19T07:31:56.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:56.957+0000 7fc019400640 1 -- 172.21.15.159:0/2621251621 wait complete. 2024-09-19T07:31:57.353 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:31:56 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:31:56.930+0000 7f0846a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:31:57.353 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:31:56 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:31:56.930+0000 7f0846a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:31:57.353 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:31:56 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:31:56.930+0000 7f0846a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:31:57.353 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:31:56 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:31:56.930+0000 7f0846a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:31:57.353 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:31:56 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:31:56.930+0000 7f0846a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:31:57.828 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-19T07:31:58.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:57 smithi159 ceph-mon[30472]: pgmap v341: 129 pgs: 129 active+clean; 584 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-19T07:31:58.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:57 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:58.178 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:57 smithi203 ceph-mon[35335]: pgmap v341: 129 pgs: 129 active+clean; 584 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-19T07:31:58.178 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:57 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:58.248 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:57 smithi189 ceph-mon[34851]: pgmap v341: 129 pgs: 129 active+clean; 584 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-19T07:31:58.248 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:57 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:58.666 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:59.134 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:58 smithi159 ceph-mon[30472]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:31:59.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:58 smithi203 ceph-mon[35335]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:31:59.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:58 smithi189 ceph-mon[34851]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:31:59.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.603+0000 7fc95c400640 1 -- 172.21.15.159:0/2241581193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc954102790 msgr2=0x7fc9541067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:59.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.603+0000 7fc95c400640 1 --2- 172.21.15.159:0/2241581193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc954102790 0x7fc9541067e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc940009900 tx=0x7fc94002a410 comp rx=0 tx=0).stop 2024-09-19T07:31:59.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.603+0000 7fc95c400640 1 -- 172.21.15.159:0/2241581193 shutdown_connections 2024-09-19T07:31:59.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.603+0000 7fc95c400640 1 --2- 172.21.15.159:0/2241581193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc954106f10 0x7fc95410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:59.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.603+0000 7fc95c400640 1 --2- 172.21.15.159:0/2241581193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc954102790 0x7fc9541067e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:59.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.603+0000 7fc95c400640 1 --2- 172.21.15.159:0/2241581193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc954101de0 0x7fc9541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:59.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.603+0000 7fc95c400640 1 -- 172.21.15.159:0/2241581193 >> 172.21.15.159:0/2241581193 conn(0x7fc9540fd8c0 msgr2=0x7fc9540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:59.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.603+0000 7fc95c400640 1 -- 172.21.15.159:0/2241581193 shutdown_connections 2024-09-19T07:31:59.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.604+0000 7fc95c400640 1 -- 172.21.15.159:0/2241581193 wait complete. 2024-09-19T07:31:59.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.605+0000 7fc95c400640 1 Processor -- start 2024-09-19T07:31:59.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.605+0000 7fc95c400640 1 -- start start 2024-09-19T07:31:59.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.605+0000 7fc95c400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc954101de0 0x7fc95419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:59.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.605+0000 7fc95c400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc954102790 0x7fc95419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:59.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.605+0000 7fc95c400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc954106f10 0x7fc95419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:59.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.605+0000 7fc95c400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc954111f70 con 0x7fc954102790 2024-09-19T07:31:59.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.605+0000 7fc95c400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc954111df0 con 0x7fc954106f10 2024-09-19T07:31:59.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.605+0000 7fc95c400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541120f0 con 0x7fc954101de0 2024-09-19T07:31:59.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.606+0000 7fc959000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc954102790 0x7fc95419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:59.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.606+0000 7fc959a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc954101de0 0x7fc95419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:59.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.606+0000 7fc959a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc954101de0 0x7fc95419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:39030/0 (socket says 172.21.15.159:39030) 2024-09-19T07:31:59.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.606+0000 7fc959a00640 1 -- 172.21.15.159:0/2210748588 learned_addr learned my addr 172.21.15.159:0/2210748588 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:31:59.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.606+0000 7fc95a400640 1 --2- 172.21.15.159:0/2210748588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc954106f10 0x7fc95419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:59.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.606+0000 7fc959a00640 1 -- 172.21.15.159:0/2210748588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc954106f10 msgr2=0x7fc95419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:59.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.606+0000 7fc959a00640 1 --2- 172.21.15.159:0/2210748588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc954106f10 0x7fc95419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:59.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.606+0000 7fc959a00640 1 -- 172.21.15.159:0/2210748588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc954102790 msgr2=0x7fc95419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:59.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.606+0000 7fc959a00640 1 --2- 172.21.15.159:0/2210748588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc954102790 0x7fc95419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:59.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.606+0000 7fc959a00640 1 -- 172.21.15.159:0/2210748588 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95419f640 con 0x7fc954101de0 2024-09-19T07:31:59.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.606+0000 7fc95a400640 1 --2- 172.21.15.159:0/2210748588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc954106f10 0x7fc95419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:31:59.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.607+0000 7fc959a00640 1 --2- 172.21.15.159:0/2210748588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc954101de0 0x7fc95419a690 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc94800ea60 tx=0x7fc94800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:59.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.607+0000 7fc952000640 1 -- 172.21.15.159:0/2210748588 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94800cde0 con 0x7fc954101de0 2024-09-19T07:31:59.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.607+0000 7fc952000640 1 -- 172.21.15.159:0/2210748588 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9480035f0 con 0x7fc954101de0 2024-09-19T07:31:59.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.607+0000 7fc95c400640 1 -- 172.21.15.159:0/2210748588 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc95419f930 con 0x7fc954101de0 2024-09-19T07:31:59.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.607+0000 7fc952000640 1 -- 172.21.15.159:0/2210748588 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948010670 con 0x7fc954101de0 2024-09-19T07:31:59.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.607+0000 7fc95c400640 1 -- 172.21.15.159:0/2210748588 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9541a7210 con 0x7fc954101de0 2024-09-19T07:31:59.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.608+0000 7fc95c400640 1 -- 172.21.15.159:0/2210748588 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9541021c0 con 0x7fc954101de0 2024-09-19T07:31:59.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.610+0000 7fc952000640 1 -- 172.21.15.159:0/2210748588 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc9480037a0 con 0x7fc954101de0 2024-09-19T07:31:59.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.611+0000 7fc952000640 1 --2- 172.21.15.159:0/2210748588 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc93c0a32d0 0x7fc93c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:59.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.611+0000 7fc952000640 1 -- 172.21.15.159:0/2210748588 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fc9480d01c0 con 0x7fc954101de0 2024-09-19T07:31:59.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.612+0000 7fc959000640 1 --2- 172.21.15.159:0/2210748588 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc93c0a32d0 0x7fc93c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:59.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.612+0000 7fc959000640 1 --2- 172.21.15.159:0/2210748588 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc93c0a32d0 0x7fc93c0a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc940009230 tx=0x7fc9400058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:59.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.615+0000 7fc952000640 1 -- 172.21.15.159:0/2210748588 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc94809c9c0 con 0x7fc954101de0 2024-09-19T07:31:59.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.843+0000 7fc95c400640 1 -- 172.21.15.159:0/2210748588 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc95410de00 con 0x7fc93c0a32d0 2024-09-19T07:31:59.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.866+0000 7fc952000640 1 -- 172.21.15.159:0/2210748588 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc95410de00 con 0x7fc93c0a32d0 2024-09-19T07:31:59.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.869+0000 7fc95c400640 1 -- 172.21.15.159:0/2210748588 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc93c0a32d0 msgr2=0x7fc93c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:59.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.869+0000 7fc95c400640 1 --2- 172.21.15.159:0/2210748588 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc93c0a32d0 0x7fc93c0a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc940009230 tx=0x7fc9400058d0 comp rx=0 tx=0).stop 2024-09-19T07:31:59.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.869+0000 7fc95c400640 1 -- 172.21.15.159:0/2210748588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc954101de0 msgr2=0x7fc95419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:59.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.869+0000 7fc95c400640 1 --2- 172.21.15.159:0/2210748588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc954101de0 0x7fc95419a690 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc94800ea60 tx=0x7fc94800ef20 comp rx=0 tx=0).stop 2024-09-19T07:31:59.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.870+0000 7fc95c400640 1 -- 172.21.15.159:0/2210748588 shutdown_connections 2024-09-19T07:31:59.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.870+0000 7fc95c400640 1 --2- 172.21.15.159:0/2210748588 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc93c0a32d0 0x7fc93c0a5790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:59.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.870+0000 7fc95c400640 1 --2- 172.21.15.159:0/2210748588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc954106f10 0x7fc95419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:59.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.870+0000 7fc95c400640 1 --2- 172.21.15.159:0/2210748588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc954102790 0x7fc95419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:59.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.870+0000 7fc95c400640 1 --2- 172.21.15.159:0/2210748588 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc954101de0 0x7fc95419a690 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:59.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.870+0000 7fc95c400640 1 -- 172.21.15.159:0/2210748588 >> 172.21.15.159:0/2210748588 conn(0x7fc9540fd8c0 msgr2=0x7fc9540fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:59.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.870+0000 7fc95c400640 1 -- 172.21.15.159:0/2210748588 shutdown_connections 2024-09-19T07:31:59.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:31:59.870+0000 7fc95c400640 1 -- 172.21.15.159:0/2210748588 wait complete. 2024-09-19T07:32:00.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:59 smithi159 ceph-mon[30472]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-19T07:32:00.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:59 smithi159 ceph-mon[30472]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-19T07:32:00.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:59 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:00.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:59 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:00.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:59 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:00.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:31:59 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:00.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:59 smithi203 ceph-mon[35335]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-19T07:32:00.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:59 smithi203 ceph-mon[35335]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-19T07:32:00.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:59 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:00.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:59 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:00.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:59 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:00.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:31:59 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:00.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:59 smithi189 ceph-mon[34851]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-19T07:32:00.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:59 smithi189 ceph-mon[34851]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-19T07:32:00.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:59 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:00.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:59 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:00.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:59 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:00.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:31:59 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:00.671 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-19T07:32:01.068 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:01.215 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:00 smithi159 ceph-mon[30472]: from='client.24730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:01.215 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:00 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:01.215 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:00 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:01.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:00 smithi203 ceph-mon[35335]: from='client.24730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:01.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:00 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:01.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:00 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:01.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:00 smithi189 ceph-mon[34851]: from='client.24730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:01.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:00 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:01.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:00 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:01.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.724+0000 7fdfc8200640 1 -- 172.21.15.159:0/2834550810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc0074620 msgr2=0x7fdfc0074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:01.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.724+0000 7fdfc8200640 1 --2- 172.21.15.159:0/2834550810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc0074620 0x7fdfc0074a00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009860 tx=0x7fdfb402a3d0 comp rx=0 tx=0).stop 2024-09-19T07:32:01.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.724+0000 7fdfc8200640 1 -- 172.21.15.159:0/2834550810 shutdown_connections 2024-09-19T07:32:01.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.724+0000 7fdfc8200640 1 --2- 172.21.15.159:0/2834550810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfc010e3b0 0x7fdfc0110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:01.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.724+0000 7fdfc8200640 1 --2- 172.21.15.159:0/2834550810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfc0074f40 0x7fdfc010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:01.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.724+0000 7fdfc8200640 1 --2- 172.21.15.159:0/2834550810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc0074620 0x7fdfc0074a00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:01.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.724+0000 7fdfc8200640 1 -- 172.21.15.159:0/2834550810 >> 172.21.15.159:0/2834550810 conn(0x7fdfc00fd8b0 msgr2=0x7fdfc00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:01.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.724+0000 7fdfc8200640 1 -- 172.21.15.159:0/2834550810 shutdown_connections 2024-09-19T07:32:01.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.724+0000 7fdfc8200640 1 -- 172.21.15.159:0/2834550810 wait complete. 2024-09-19T07:32:01.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.725+0000 7fdfc8200640 1 Processor -- start 2024-09-19T07:32:01.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.725+0000 7fdfc8200640 1 -- start start 2024-09-19T07:32:01.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.726+0000 7fdfc8200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc0074620 0x7fdfc019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:01.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.726+0000 7fdfc8200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfc0074f40 0x7fdfc019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:01.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.726+0000 7fdfc8200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfc010e3b0 0x7fdfc019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:01.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.726+0000 7fdfc8200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc0114160 con 0x7fdfc0074f40 2024-09-19T07:32:01.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.726+0000 7fdfc8200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc0113fe0 con 0x7fdfc010e3b0 2024-09-19T07:32:01.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.726+0000 7fdfc8200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc01142e0 con 0x7fdfc0074620 2024-09-19T07:32:01.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.726+0000 7fdfc5800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc0074620 0x7fdfc019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:01.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.726+0000 7fdfc5800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc0074620 0x7fdfc019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:39042/0 (socket says 172.21.15.159:39042) 2024-09-19T07:32:01.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.726+0000 7fdfc5800640 1 -- 172.21.15.159:0/98400127 learned_addr learned my addr 172.21.15.159:0/98400127 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:01.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.726+0000 7fdfc4e00640 1 --2- 172.21.15.159:0/98400127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfc0074f40 0x7fdfc019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:01.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.726+0000 7fdfc6200640 1 --2- 172.21.15.159:0/98400127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfc010e3b0 0x7fdfc019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:01.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.726+0000 7fdfc5800640 1 -- 172.21.15.159:0/98400127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfc010e3b0 msgr2=0x7fdfc019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:01.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.726+0000 7fdfc5800640 1 --2- 172.21.15.159:0/98400127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfc010e3b0 0x7fdfc019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:01.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.727+0000 7fdfc5800640 1 -- 172.21.15.159:0/98400127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfc0074f40 msgr2=0x7fdfc019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:01.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.727+0000 7fdfc5800640 1 --2- 172.21.15.159:0/98400127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfc0074f40 0x7fdfc019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:01.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.727+0000 7fdfc5800640 1 -- 172.21.15.159:0/98400127 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc019f650 con 0x7fdfc0074620 2024-09-19T07:32:01.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.727+0000 7fdfc4e00640 1 --2- 172.21.15.159:0/98400127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfc0074f40 0x7fdfc019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:32:01.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.727+0000 7fdfc5800640 1 --2- 172.21.15.159:0/98400127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc0074620 0x7fdfc019a6a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdfb402a8f0 tx=0x7fdfb4030650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:01.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.727+0000 7fdfbe000640 1 -- 172.21.15.159:0/98400127 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4040020 con 0x7fdfc0074620 2024-09-19T07:32:01.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.727+0000 7fdfbe000640 1 -- 172.21.15.159:0/98400127 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb4005510 con 0x7fdfc0074620 2024-09-19T07:32:01.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.727+0000 7fdfc8200640 1 -- 172.21.15.159:0/98400127 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc019f8e0 con 0x7fdfc0074620 2024-09-19T07:32:01.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.727+0000 7fdfbe000640 1 -- 172.21.15.159:0/98400127 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb40046c0 con 0x7fdfc0074620 2024-09-19T07:32:01.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.727+0000 7fdfc8200640 1 -- 172.21.15.159:0/98400127 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc01a7220 con 0x7fdfc0074620 2024-09-19T07:32:01.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.729+0000 7fdfc8200640 1 -- 172.21.15.159:0/98400127 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc0074a00 con 0x7fdfc0074620 2024-09-19T07:32:01.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.731+0000 7fdfbe000640 1 -- 172.21.15.159:0/98400127 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fdfb4043020 con 0x7fdfc0074620 2024-09-19T07:32:01.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.732+0000 7fdfbe000640 1 --2- 172.21.15.159:0/98400127 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdfa80a3350 0x7fdfa80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:01.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.732+0000 7fdfbe000640 1 -- 172.21.15.159:0/98400127 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fdfb40f10a0 con 0x7fdfc0074620 2024-09-19T07:32:01.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.732+0000 7fdfc4e00640 1 --2- 172.21.15.159:0/98400127 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdfa80a3350 0x7fdfa80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:01.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.733+0000 7fdfc4e00640 1 --2- 172.21.15.159:0/98400127 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdfa80a3350 0x7fdfa80a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00fff20 tx=0x7fdfac009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:01.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.734+0000 7fdfbe000640 1 -- 172.21.15.159:0/98400127 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdfb404a020 con 0x7fdfc0074620 2024-09-19T07:32:01.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.954+0000 7fdfc8200640 1 -- 172.21.15.159:0/98400127 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fdfc0062630 con 0x7fdfa80a3350 2024-09-19T07:32:01.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.959+0000 7fdfbe000640 1 -- 172.21.15.159:0/98400127 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fdfc0062630 con 0x7fdfa80a3350 2024-09-19T07:32:01.960 INFO:teuthology.orchestra.run.smithi159.stdout:/volumes/qa/sv_0/60f9a781-35c8-4b91-acf4-9c29b905d53e 2024-09-19T07:32:01.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.961+0000 7fdfc8200640 1 -- 172.21.15.159:0/98400127 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdfa80a3350 msgr2=0x7fdfa80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:01.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.961+0000 7fdfc8200640 1 --2- 172.21.15.159:0/98400127 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdfa80a3350 0x7fdfa80a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00fff20 tx=0x7fdfac009260 comp rx=0 tx=0).stop 2024-09-19T07:32:01.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.961+0000 7fdfc8200640 1 -- 172.21.15.159:0/98400127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc0074620 msgr2=0x7fdfc019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:01.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.961+0000 7fdfc8200640 1 --2- 172.21.15.159:0/98400127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc0074620 0x7fdfc019a6a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdfb402a8f0 tx=0x7fdfb4030650 comp rx=0 tx=0).stop 2024-09-19T07:32:01.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.961+0000 7fdfc8200640 1 -- 172.21.15.159:0/98400127 shutdown_connections 2024-09-19T07:32:01.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.961+0000 7fdfc8200640 1 --2- 172.21.15.159:0/98400127 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdfa80a3350 0x7fdfa80a5810 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:01.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.961+0000 7fdfc8200640 1 --2- 172.21.15.159:0/98400127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfc010e3b0 0x7fdfc019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:01.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.961+0000 7fdfc8200640 1 --2- 172.21.15.159:0/98400127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfc0074f40 0x7fdfc019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:01.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.961+0000 7fdfc8200640 1 --2- 172.21.15.159:0/98400127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc0074620 0x7fdfc019a6a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:01.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.961+0000 7fdfc8200640 1 -- 172.21.15.159:0/98400127 >> 172.21.15.159:0/98400127 conn(0x7fdfc00fd8b0 msgr2=0x7fdfc00ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:01.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.962+0000 7fdfc8200640 1 -- 172.21.15.159:0/98400127 shutdown_connections 2024-09-19T07:32:01.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:01.962+0000 7fdfc8200640 1 -- 172.21.15.159:0/98400127 wait complete. 2024-09-19T07:32:02.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:01 smithi159 ceph-mon[30472]: pgmap v343: 129 pgs: 129 active+clean; 584 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-19T07:32:02.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:02.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:32:02.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:01 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:02.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:01 smithi203 ceph-mon[35335]: pgmap v343: 129 pgs: 129 active+clean; 584 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-19T07:32:02.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:02.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:32:02.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:01 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:02.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:01 smithi189 ceph-mon[34851]: pgmap v343: 129 pgs: 129 active+clean; 584 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-19T07:32:02.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:02.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:32:02.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:01 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:32:02.801 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-19T07:32:03.179 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:03.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:02 smithi203 ceph-mon[35335]: from='client.24736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:03.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:02 smithi203 ceph-mon[35335]: pgmap v344: 129 pgs: 129 active+clean; 605 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-09-19T07:32:03.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:02 smithi189 ceph-mon[34851]: from='client.24736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:03.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:02 smithi189 ceph-mon[34851]: pgmap v344: 129 pgs: 129 active+clean; 605 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-09-19T07:32:03.319 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:02 smithi159 ceph-mon[30472]: from='client.24736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:03.319 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:02 smithi159 ceph-mon[30472]: pgmap v344: 129 pgs: 129 active+clean; 605 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-09-19T07:32:03.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.803+0000 7fd566c00640 1 -- 172.21.15.159:0/2056089158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd560102b90 msgr2=0x7fd560102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:03.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.803+0000 7fd566c00640 1 --2- 172.21.15.159:0/2056089158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd560102b90 0x7fd560102f70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd554009900 tx=0x7fd55402a480 comp rx=0 tx=0).stop 2024-09-19T07:32:03.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.803+0000 7fd566c00640 1 -- 172.21.15.159:0/2056089158 shutdown_connections 2024-09-19T07:32:03.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.803+0000 7fd566c00640 1 --2- 172.21.15.159:0/2056089158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd56010bfc0 0x7fd56010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:03.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.803+0000 7fd566c00640 1 --2- 172.21.15.159:0/2056089158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5601034b0 0x7fd56010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:03.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.803+0000 7fd566c00640 1 --2- 172.21.15.159:0/2056089158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd560102b90 0x7fd560102f70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:03.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.803+0000 7fd566c00640 1 -- 172.21.15.159:0/2056089158 >> 172.21.15.159:0/2056089158 conn(0x7fd5600fc610 msgr2=0x7fd5600fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:03.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.803+0000 7fd566c00640 1 -- 172.21.15.159:0/2056089158 shutdown_connections 2024-09-19T07:32:03.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.803+0000 7fd566c00640 1 -- 172.21.15.159:0/2056089158 wait complete. 2024-09-19T07:32:03.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.804+0000 7fd566c00640 1 Processor -- start 2024-09-19T07:32:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.804+0000 7fd566c00640 1 -- start start 2024-09-19T07:32:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.805+0000 7fd566c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd560102b90 0x7fd56019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.805+0000 7fd566c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5601034b0 0x7fd56019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.805+0000 7fd566c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd56010bfc0 0x7fd56019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.805+0000 7fd566c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd560111dc0 con 0x7fd560102b90 2024-09-19T07:32:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.805+0000 7fd566c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd560111c40 con 0x7fd56010bfc0 2024-09-19T07:32:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.805+0000 7fd566c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd560111f40 con 0x7fd5601034b0 2024-09-19T07:32:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.805+0000 7fd564e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5601034b0 0x7fd56019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:03.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.805+0000 7fd564e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5601034b0 0x7fd56019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:52532/0 (socket says 172.21.15.159:52532) 2024-09-19T07:32:03.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.805+0000 7fd564e00640 1 -- 172.21.15.159:0/3412919637 learned_addr learned my addr 172.21.15.159:0/3412919637 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:03.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.805+0000 7fd566200640 1 --2- 172.21.15.159:0/3412919637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd56010bfc0 0x7fd56019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:03.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.805+0000 7fd565800640 1 --2- 172.21.15.159:0/3412919637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd560102b90 0x7fd56019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:03.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.805+0000 7fd564e00640 1 -- 172.21.15.159:0/3412919637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd56010bfc0 msgr2=0x7fd56019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:03.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.805+0000 7fd564e00640 1 --2- 172.21.15.159:0/3412919637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd56010bfc0 0x7fd56019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:03.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.805+0000 7fd564e00640 1 -- 172.21.15.159:0/3412919637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd560102b90 msgr2=0x7fd56019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:03.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.805+0000 7fd564e00640 1 --2- 172.21.15.159:0/3412919637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd560102b90 0x7fd56019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:03.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.805+0000 7fd564e00640 1 -- 172.21.15.159:0/3412919637 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56019f470 con 0x7fd5601034b0 2024-09-19T07:32:03.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.805+0000 7fd565800640 1 --2- 172.21.15.159:0/3412919637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd560102b90 0x7fd56019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:32:03.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.806+0000 7fd564e00640 1 --2- 172.21.15.159:0/3412919637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5601034b0 0x7fd56019aa00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00e9f0 tx=0x7fd54c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.806+0000 7fd55e000640 1 -- 172.21.15.159:0/3412919637 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c00ccc0 con 0x7fd5601034b0 2024-09-19T07:32:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.806+0000 7fd566c00640 1 -- 172.21.15.159:0/3412919637 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56019f760 con 0x7fd5601034b0 2024-09-19T07:32:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.806+0000 7fd55e000640 1 -- 172.21.15.159:0/3412919637 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd54c010910 con 0x7fd5601034b0 2024-09-19T07:32:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.806+0000 7fd55e000640 1 -- 172.21.15.159:0/3412919637 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c018920 con 0x7fd5601034b0 2024-09-19T07:32:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.806+0000 7fd566c00640 1 -- 172.21.15.159:0/3412919637 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5601a7040 con 0x7fd5601034b0 2024-09-19T07:32:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.807+0000 7fd566c00640 1 -- 172.21.15.159:0/3412919637 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd560102f70 con 0x7fd5601034b0 2024-09-19T07:32:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.809+0000 7fd55e000640 1 -- 172.21.15.159:0/3412919637 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd54c010430 con 0x7fd5601034b0 2024-09-19T07:32:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.811+0000 7fd55e000640 1 --2- 172.21.15.159:0/3412919637 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd5480a3350 0x7fd5480a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.811+0000 7fd55e000640 1 -- 172.21.15.159:0/3412919637 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fd54c0d00d0 con 0x7fd5601034b0 2024-09-19T07:32:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.812+0000 7fd565800640 1 --2- 172.21.15.159:0/3412919637 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd5480a3350 0x7fd5480a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.812+0000 7fd565800640 1 --2- 172.21.15.159:0/3412919637 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd5480a3350 0x7fd5480a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd554000c00 tx=0x7fd554032c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:03.815+0000 7fd55e000640 1 -- 172.21.15.159:0/3412919637 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd54c09c8d0 con 0x7fd5601034b0 2024-09-19T07:32:04.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:04.025+0000 7fd566c00640 1 -- 172.21.15.159:0/3412919637 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd5600fe930 con 0x7fd5480a3350 2024-09-19T07:32:04.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:04.052+0000 7fd55e000640 1 -- 172.21.15.159:0/3412919637 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd5600fe930 con 0x7fd5480a3350 2024-09-19T07:32:04.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:04.054+0000 7fd566c00640 1 -- 172.21.15.159:0/3412919637 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd5480a3350 msgr2=0x7fd5480a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:04.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:04.054+0000 7fd566c00640 1 --2- 172.21.15.159:0/3412919637 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd5480a3350 0x7fd5480a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd554000c00 tx=0x7fd554032c70 comp rx=0 tx=0).stop 2024-09-19T07:32:04.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:04.054+0000 7fd566c00640 1 -- 172.21.15.159:0/3412919637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5601034b0 msgr2=0x7fd56019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:04.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:04.054+0000 7fd566c00640 1 --2- 172.21.15.159:0/3412919637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5601034b0 0x7fd56019aa00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00e9f0 tx=0x7fd54c00eeb0 comp rx=0 tx=0).stop 2024-09-19T07:32:04.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:04.054+0000 7fd566c00640 1 -- 172.21.15.159:0/3412919637 shutdown_connections 2024-09-19T07:32:04.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:04.054+0000 7fd566c00640 1 --2- 172.21.15.159:0/3412919637 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd5480a3350 0x7fd5480a5810 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:04.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:04.054+0000 7fd566c00640 1 --2- 172.21.15.159:0/3412919637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd56010bfc0 0x7fd56019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:04.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:04.054+0000 7fd566c00640 1 --2- 172.21.15.159:0/3412919637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5601034b0 0x7fd56019aa00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:04.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:04.054+0000 7fd566c00640 1 --2- 172.21.15.159:0/3412919637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd560102b90 0x7fd56019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:04.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:04.054+0000 7fd566c00640 1 -- 172.21.15.159:0/3412919637 >> 172.21.15.159:0/3412919637 conn(0x7fd5600fc610 msgr2=0x7fd5600fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:04.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:04.055+0000 7fd566c00640 1 -- 172.21.15.159:0/3412919637 shutdown_connections 2024-09-19T07:32:04.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:04.055+0000 7fd566c00640 1 -- 172.21.15.159:0/3412919637 wait complete. 2024-09-19T07:32:04.901 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-19T07:32:05.276 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:05.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.917+0000 7fdf39a00640 1 -- 172.21.15.159:0/986346765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf34101de0 msgr2=0x7fdf341021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:05.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.917+0000 7fdf39a00640 1 --2- 172.21.15.159:0/986346765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf34101de0 0x7fdf341021c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf24009900 tx=0x7fdf2402a410 comp rx=0 tx=0).stop 2024-09-19T07:32:05.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.917+0000 7fdf39a00640 1 -- 172.21.15.159:0/986346765 shutdown_connections 2024-09-19T07:32:05.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.917+0000 7fdf39a00640 1 --2- 172.21.15.159:0/986346765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf34106f10 0x7fdf3410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:05.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.917+0000 7fdf39a00640 1 --2- 172.21.15.159:0/986346765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf34102790 0x7fdf341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:05.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.917+0000 7fdf39a00640 1 --2- 172.21.15.159:0/986346765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf34101de0 0x7fdf341021c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:05.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.917+0000 7fdf39a00640 1 -- 172.21.15.159:0/986346765 >> 172.21.15.159:0/986346765 conn(0x7fdf340fd8c0 msgr2=0x7fdf340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:05.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.918+0000 7fdf39a00640 1 -- 172.21.15.159:0/986346765 shutdown_connections 2024-09-19T07:32:05.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.918+0000 7fdf39a00640 1 -- 172.21.15.159:0/986346765 wait complete. 2024-09-19T07:32:05.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.918+0000 7fdf39a00640 1 Processor -- start 2024-09-19T07:32:05.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.919+0000 7fdf39a00640 1 -- start start 2024-09-19T07:32:05.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.919+0000 7fdf39a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf34101de0 0x7fdf3419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:05.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.919+0000 7fdf39a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf34102790 0x7fdf3419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:05.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.919+0000 7fdf39a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf34106f10 0x7fdf3419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:05.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.919+0000 7fdf39a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdf34111f70 con 0x7fdf34106f10 2024-09-19T07:32:05.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.919+0000 7fdf39a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdf34111df0 con 0x7fdf34102790 2024-09-19T07:32:05.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.919+0000 7fdf39a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341120f0 con 0x7fdf34101de0 2024-09-19T07:32:05.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.919+0000 7fdf2a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf34102790 0x7fdf3419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:05.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.919+0000 7fdf32a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf34101de0 0x7fdf3419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:05.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.919+0000 7fdf2a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf34102790 0x7fdf3419ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:46702/0 (socket says 172.21.15.159:46702) 2024-09-19T07:32:05.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.919+0000 7fdf2a000640 1 -- 172.21.15.159:0/2263330971 learned_addr learned my addr 172.21.15.159:0/2263330971 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:05.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.919+0000 7fdf33400640 1 --2- 172.21.15.159:0/2263330971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf34106f10 0x7fdf3419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:05.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.920+0000 7fdf2a000640 1 -- 172.21.15.159:0/2263330971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf34101de0 msgr2=0x7fdf3419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:05.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.920+0000 7fdf2a000640 1 --2- 172.21.15.159:0/2263330971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf34101de0 0x7fdf3419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:05.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.920+0000 7fdf2a000640 1 -- 172.21.15.159:0/2263330971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf34106f10 msgr2=0x7fdf3419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:05.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.920+0000 7fdf2a000640 1 --2- 172.21.15.159:0/2263330971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf34106f10 0x7fdf3419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:05.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.920+0000 7fdf2a000640 1 -- 172.21.15.159:0/2263330971 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3419f600 con 0x7fdf34102790 2024-09-19T07:32:05.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.920+0000 7fdf2a000640 1 --2- 172.21.15.159:0/2263330971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf34102790 0x7fdf3419ab90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00e9f0 tx=0x7fdf1c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:05.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.920+0000 7fdf2be00640 1 -- 172.21.15.159:0/2263330971 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c00cd80 con 0x7fdf34102790 2024-09-19T07:32:05.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.920+0000 7fdf2be00640 1 -- 172.21.15.159:0/2263330971 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf1c005500 con 0x7fdf34102790 2024-09-19T07:32:05.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.920+0000 7fdf39a00640 1 -- 172.21.15.159:0/2263330971 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3419f8f0 con 0x7fdf34102790 2024-09-19T07:32:05.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.920+0000 7fdf2be00640 1 -- 172.21.15.159:0/2263330971 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c010690 con 0x7fdf34102790 2024-09-19T07:32:05.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.920+0000 7fdf39a00640 1 -- 172.21.15.159:0/2263330971 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf341a71d0 con 0x7fdf34102790 2024-09-19T07:32:05.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.921+0000 7fdf39a00640 1 -- 172.21.15.159:0/2263330971 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf341021c0 con 0x7fdf34102790 2024-09-19T07:32:05.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.923+0000 7fdf2be00640 1 -- 172.21.15.159:0/2263330971 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fdf1c0046c0 con 0x7fdf34102790 2024-09-19T07:32:05.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.925+0000 7fdf2be00640 1 --2- 172.21.15.159:0/2263330971 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdf180a32d0 0x7fdf180a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:05.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.925+0000 7fdf2be00640 1 -- 172.21.15.159:0/2263330971 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fdf1c0d0500 con 0x7fdf34102790 2024-09-19T07:32:05.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.925+0000 7fdf32a00640 1 --2- 172.21.15.159:0/2263330971 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdf180a32d0 0x7fdf180a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:05.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.926+0000 7fdf32a00640 1 --2- 172.21.15.159:0/2263330971 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdf180a32d0 0x7fdf180a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdf24009230 tx=0x7fdf240058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:05.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:05.927+0000 7fdf2be00640 1 -- 172.21.15.159:0/2263330971 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdf1c09cd00 con 0x7fdf34102790 2024-09-19T07:32:06.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:05 smithi203 ceph-mon[35335]: from='client.24742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:06.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:05 smithi203 ceph-mon[35335]: pgmap v345: 129 pgs: 129 active+clean; 605 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-19T07:32:06.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:05 smithi189 ceph-mon[34851]: from='client.24742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:06.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:05 smithi189 ceph-mon[34851]: pgmap v345: 129 pgs: 129 active+clean; 605 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-19T07:32:06.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:06.128+0000 7fdf39a00640 1 -- 172.21.15.159:0/2263330971 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fdf34075430 con 0x7fdf180a32d0 2024-09-19T07:32:06.130 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:05 smithi159 ceph-mon[30472]: from='client.24742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:06.130 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:05 smithi159 ceph-mon[30472]: pgmap v345: 129 pgs: 129 active+clean; 605 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-19T07:32:06.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:06.133+0000 7fdf2be00640 1 -- 172.21.15.159:0/2263330971 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fdf34075430 con 0x7fdf180a32d0 2024-09-19T07:32:06.134 INFO:teuthology.orchestra.run.smithi159.stdout:/volumes/qa/sv_1/7962f356-4b90-4a53-adf1-2e770a41cab4 2024-09-19T07:32:06.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:06.135+0000 7fdf39a00640 1 -- 172.21.15.159:0/2263330971 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdf180a32d0 msgr2=0x7fdf180a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:06.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:06.135+0000 7fdf39a00640 1 --2- 172.21.15.159:0/2263330971 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdf180a32d0 0x7fdf180a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdf24009230 tx=0x7fdf240058d0 comp rx=0 tx=0).stop 2024-09-19T07:32:06.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:06.135+0000 7fdf39a00640 1 -- 172.21.15.159:0/2263330971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf34102790 msgr2=0x7fdf3419ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:06.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:06.135+0000 7fdf39a00640 1 --2- 172.21.15.159:0/2263330971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf34102790 0x7fdf3419ab90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00e9f0 tx=0x7fdf1c00edf0 comp rx=0 tx=0).stop 2024-09-19T07:32:06.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:06.135+0000 7fdf39a00640 1 -- 172.21.15.159:0/2263330971 shutdown_connections 2024-09-19T07:32:06.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:06.135+0000 7fdf39a00640 1 --2- 172.21.15.159:0/2263330971 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdf180a32d0 0x7fdf180a5790 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:06.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:06.135+0000 7fdf39a00640 1 --2- 172.21.15.159:0/2263330971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf34106f10 0x7fdf3419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:06.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:06.135+0000 7fdf39a00640 1 --2- 172.21.15.159:0/2263330971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf34102790 0x7fdf3419ab90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:06.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:06.135+0000 7fdf39a00640 1 --2- 172.21.15.159:0/2263330971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf34101de0 0x7fdf3419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:06.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:06.135+0000 7fdf39a00640 1 -- 172.21.15.159:0/2263330971 >> 172.21.15.159:0/2263330971 conn(0x7fdf340fd8c0 msgr2=0x7fdf340feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:06.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:06.135+0000 7fdf39a00640 1 -- 172.21.15.159:0/2263330971 shutdown_connections 2024-09-19T07:32:06.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:06.136+0000 7fdf39a00640 1 -- 172.21.15.159:0/2263330971 wait complete. 2024-09-19T07:32:06.929 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:32:07.299 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:07.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.946+0000 7f50faa00640 1 -- 172.21.15.159:0/1101767759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50f410e3c0 msgr2=0x7f50f4110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:07.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.946+0000 7f50faa00640 1 --2- 172.21.15.159:0/1101767759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50f410e3c0 0x7f50f4110810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009ca0 tx=0x7f50ec02a950 comp rx=0 tx=0).stop 2024-09-19T07:32:07.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.946+0000 7f50faa00640 1 -- 172.21.15.159:0/1101767759 shutdown_connections 2024-09-19T07:32:07.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.946+0000 7f50faa00640 1 --2- 172.21.15.159:0/1101767759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50f410e3c0 0x7f50f4110810 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:07.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.946+0000 7f50faa00640 1 --2- 172.21.15.159:0/1101767759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50f4074f40 0x7f50f410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:07.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.946+0000 7f50faa00640 1 --2- 172.21.15.159:0/1101767759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50f4074620 0x7f50f4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:07.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.946+0000 7f50faa00640 1 -- 172.21.15.159:0/1101767759 >> 172.21.15.159:0/1101767759 conn(0x7f50f40fd8c0 msgr2=0x7f50f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:07.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.946+0000 7f50faa00640 1 -- 172.21.15.159:0/1101767759 shutdown_connections 2024-09-19T07:32:07.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.946+0000 7f50faa00640 1 -- 172.21.15.159:0/1101767759 wait complete. 2024-09-19T07:32:07.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.947+0000 7f50faa00640 1 Processor -- start 2024-09-19T07:32:07.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.947+0000 7f50faa00640 1 -- start start 2024-09-19T07:32:07.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.947+0000 7f50faa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50f4074620 0x7f50f419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:07.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.947+0000 7f50faa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50f4074f40 0x7f50f419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:07.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.948+0000 7f50faa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50f410e3c0 0x7f50f419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:07.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.948+0000 7f50faa00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4114150 con 0x7f50f410e3c0 2024-09-19T07:32:07.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.948+0000 7f50faa00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4113fd0 con 0x7f50f4074f40 2024-09-19T07:32:07.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.948+0000 7f50faa00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41142d0 con 0x7f50f4074620 2024-09-19T07:32:07.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.948+0000 7f50f8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50f410e3c0 0x7f50f419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:07.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.948+0000 7f50f8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50f410e3c0 0x7f50f419ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44766/0 (socket says 172.21.15.159:44766) 2024-09-19T07:32:07.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.948+0000 7f50f8a00640 1 -- 172.21.15.159:0/2173434015 learned_addr learned my addr 172.21.15.159:0/2173434015 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:07.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.948+0000 7f50f3400640 1 --2- 172.21.15.159:0/2173434015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50f4074f40 0x7f50f419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:07.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.948+0000 7f50f3e00640 1 --2- 172.21.15.159:0/2173434015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50f4074620 0x7f50f419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:07.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.948+0000 7f50f3e00640 1 -- 172.21.15.159:0/2173434015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50f4074f40 msgr2=0x7f50f419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:07.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.948+0000 7f50f3e00640 1 --2- 172.21.15.159:0/2173434015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50f4074f40 0x7f50f419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:07.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.948+0000 7f50f3e00640 1 -- 172.21.15.159:0/2173434015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50f410e3c0 msgr2=0x7f50f419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:07.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.948+0000 7f50f3e00640 1 --2- 172.21.15.159:0/2173434015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50f410e3c0 0x7f50f419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:07.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.948+0000 7f50f3e00640 1 -- 172.21.15.159:0/2173434015 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f419f640 con 0x7f50f4074620 2024-09-19T07:32:07.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.949+0000 7f50f3e00640 1 --2- 172.21.15.159:0/2173434015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50f4074620 0x7f50f419a690 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50e400e9f0 tx=0x7f50e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:07.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.949+0000 7f50f0c00640 1 -- 172.21.15.159:0/2173434015 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e400ccc0 con 0x7f50f4074620 2024-09-19T07:32:07.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.949+0000 7f50f0c00640 1 -- 172.21.15.159:0/2173434015 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50e4010910 con 0x7f50f4074620 2024-09-19T07:32:07.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.949+0000 7f50f0c00640 1 -- 172.21.15.159:0/2173434015 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4018960 con 0x7f50f4074620 2024-09-19T07:32:07.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.949+0000 7f50faa00640 1 -- 172.21.15.159:0/2173434015 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f419f930 con 0x7f50f4074620 2024-09-19T07:32:07.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.949+0000 7f50faa00640 1 -- 172.21.15.159:0/2173434015 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f41a7210 con 0x7f50f4074620 2024-09-19T07:32:07.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.950+0000 7f50faa00640 1 -- 172.21.15.159:0/2173434015 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f4074a00 con 0x7f50f4074620 2024-09-19T07:32:07.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.952+0000 7f50f0c00640 1 -- 172.21.15.159:0/2173434015 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f50e4010ac0 con 0x7f50f4074620 2024-09-19T07:32:07.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.953+0000 7f50f0c00640 1 --2- 172.21.15.159:0/2173434015 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f50d80a3350 0x7f50d80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:07.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.954+0000 7f50f0c00640 1 -- 172.21.15.159:0/2173434015 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f50e40d0150 con 0x7f50f4074620 2024-09-19T07:32:07.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.954+0000 7f50f3400640 1 --2- 172.21.15.159:0/2173434015 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f50d80a3350 0x7f50d80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:07.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.954+0000 7f50f3400640 1 --2- 172.21.15.159:0/2173434015 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f50d80a3350 0x7f50d80a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f50f410f770 tx=0x7f50e000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:07.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:07.958+0000 7f50f0c00640 1 -- 172.21.15.159:0/2173434015 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f50e409c300 con 0x7f50f4074620 2024-09-19T07:32:08.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:07 smithi203 ceph-mon[35335]: from='client.24737 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:08.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:07 smithi203 ceph-mon[35335]: pgmap v346: 129 pgs: 129 active+clean; 629 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-09-19T07:32:08.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:07 smithi189 ceph-mon[34851]: from='client.24737 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:08.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:07 smithi189 ceph-mon[34851]: pgmap v346: 129 pgs: 129 active+clean; 629 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-09-19T07:32:08.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:07 smithi159 ceph-mon[30472]: from='client.24737 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:08.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:07 smithi159 ceph-mon[30472]: pgmap v346: 129 pgs: 129 active+clean; 629 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-09-19T07:32:08.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:08.166+0000 7f50faa00640 1 -- 172.21.15.159:0/2173434015 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50f419b9f0 con 0x7f50f4074620 2024-09-19T07:32:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:08.167+0000 7f50f0c00640 1 -- 172.21.15.159:0/2173434015 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11936 (secure 0 0 0) 0x7f50e40a0600 con 0x7f50f4074620 2024-09-19T07:32:08.168 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:32:08.168 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":25,"btime":"2024-09-19T07:31:54:895489+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3464449771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3464449771},{"type":"v1","addr":"172.21.15.159:6837","nonce":3464449771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2036264856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2036264856},{"type":"v1","addr":"172.21.15.159:6839","nonce":2036264856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6841/2874566405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6840","nonce":2874566405},{"type":"v1","addr":"172.21.15.159:6841","nonce":2874566405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:31:03.518840+0000","modified":"2024-09-19T07:31:53.954806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24542,"mds_1":24514,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/1340279549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1340279549},{"type":"v1","addr":"172.21.15.189:6835","nonce":1340279549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/937970878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":937970878},{"type":"v1","addr":"172.21.15.203:6835","nonce":937970878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1053443810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1053443810},{"type":"v1","addr":"172.21.15.203:6837","nonce":1053443810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.189:6837/1030825585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1030825585},{"type":"v1","addr":"172.21.15.189:6837","nonce":1030825585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.203:6839/3862781462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":3862781462},{"type":"v1","addr":"172.21.15.203:6839","nonce":3862781462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.189:6839/2361818663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2361818663},{"type":"v1","addr":"172.21.15.189:6839","nonce":2361818663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24506,24514,24542]},"id":1}]} 2024-09-19T07:32:08.169 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 25 2024-09-19T07:32:08.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:08.169+0000 7f50faa00640 1 -- 172.21.15.159:0/2173434015 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f50d80a3350 msgr2=0x7f50d80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:08.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:08.169+0000 7f50faa00640 1 --2- 172.21.15.159:0/2173434015 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f50d80a3350 0x7f50d80a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f50f410f770 tx=0x7f50e000b410 comp rx=0 tx=0).stop 2024-09-19T07:32:08.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:08.169+0000 7f50faa00640 1 -- 172.21.15.159:0/2173434015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50f4074620 msgr2=0x7f50f419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:08.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:08.169+0000 7f50faa00640 1 --2- 172.21.15.159:0/2173434015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50f4074620 0x7f50f419a690 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50e400e9f0 tx=0x7f50e400eeb0 comp rx=0 tx=0).stop 2024-09-19T07:32:08.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:08.169+0000 7f50faa00640 1 -- 172.21.15.159:0/2173434015 shutdown_connections 2024-09-19T07:32:08.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:08.169+0000 7f50faa00640 1 --2- 172.21.15.159:0/2173434015 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f50d80a3350 0x7f50d80a5810 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:08.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:08.169+0000 7f50faa00640 1 --2- 172.21.15.159:0/2173434015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50f410e3c0 0x7f50f419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:08.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:08.169+0000 7f50faa00640 1 --2- 172.21.15.159:0/2173434015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50f4074f40 0x7f50f419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:08.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:08.169+0000 7f50faa00640 1 --2- 172.21.15.159:0/2173434015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50f4074620 0x7f50f419a690 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:08.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:08.169+0000 7f50faa00640 1 -- 172.21.15.159:0/2173434015 >> 172.21.15.159:0/2173434015 conn(0x7f50f40fd8c0 msgr2=0x7f50f40fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:08.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:08.169+0000 7f50faa00640 1 -- 172.21.15.159:0/2173434015 shutdown_connections 2024-09-19T07:32:08.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:08.169+0000 7f50faa00640 1 -- 172.21.15.159:0/2173434015 wait complete. 2024-09-19T07:32:08.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:08 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2173434015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:32:08.972 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:32:09.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:08 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/2173434015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:32:09.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:08 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/2173434015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:32:09.340 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:09.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.956+0000 7f7f40e00640 1 -- 172.21.15.159:0/3599967616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f3c103e30 msgr2=0x7f7f3c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:09.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.956+0000 7f7f40e00640 1 --2- 172.21.15.159:0/3599967616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f3c103e30 0x7f7f3c104210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009900 tx=0x7f7f2c02a410 comp rx=0 tx=0).stop 2024-09-19T07:32:09.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.956+0000 7f7f40e00640 1 -- 172.21.15.159:0/3599967616 shutdown_connections 2024-09-19T07:32:09.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.956+0000 7f7f40e00640 1 --2- 172.21.15.159:0/3599967616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f3c108f60 0x7f7f3c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:09.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.956+0000 7f7f40e00640 1 --2- 172.21.15.159:0/3599967616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f3c1047e0 0x7f7f3c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:09.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.956+0000 7f7f40e00640 1 --2- 172.21.15.159:0/3599967616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f3c103e30 0x7f7f3c104210 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:09.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.956+0000 7f7f40e00640 1 -- 172.21.15.159:0/3599967616 >> 172.21.15.159:0/3599967616 conn(0x7f7f3c0ff910 msgr2=0x7f7f3c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:09.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.956+0000 7f7f40e00640 1 -- 172.21.15.159:0/3599967616 shutdown_connections 2024-09-19T07:32:09.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.956+0000 7f7f40e00640 1 -- 172.21.15.159:0/3599967616 wait complete. 2024-09-19T07:32:09.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.957+0000 7f7f40e00640 1 Processor -- start 2024-09-19T07:32:09.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.957+0000 7f7f40e00640 1 -- start start 2024-09-19T07:32:09.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.957+0000 7f7f40e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f3c103e30 0x7f7f3c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:09.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.957+0000 7f7f40e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f3c1047e0 0x7f7f3c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:09.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.958+0000 7f7f40e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f3c108f60 0x7f7f3c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:09.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.958+0000 7f7f40e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c113fc0 con 0x7f7f3c108f60 2024-09-19T07:32:09.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.958+0000 7f7f40e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c113e40 con 0x7f7f3c103e30 2024-09-19T07:32:09.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.958+0000 7f7f40e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c114140 con 0x7f7f3c1047e0 2024-09-19T07:32:09.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.958+0000 7f7f3be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f3c108f60 0x7f7f3c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:09.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.958+0000 7f7f3b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f3c103e30 0x7f7f3c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:09.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.958+0000 7f7f3be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f3c108f60 0x7f7f3c19edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44792/0 (socket says 172.21.15.159:44792) 2024-09-19T07:32:09.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.958+0000 7f7f3be00640 1 -- 172.21.15.159:0/2292883561 learned_addr learned my addr 172.21.15.159:0/2292883561 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:09.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.958+0000 7f7f3aa00640 1 --2- 172.21.15.159:0/2292883561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f3c1047e0 0x7f7f3c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:09.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.958+0000 7f7f3be00640 1 -- 172.21.15.159:0/2292883561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f3c1047e0 msgr2=0x7f7f3c19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:09.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.958+0000 7f7f3be00640 1 --2- 172.21.15.159:0/2292883561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f3c1047e0 0x7f7f3c19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:09.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.958+0000 7f7f3be00640 1 -- 172.21.15.159:0/2292883561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f3c103e30 msgr2=0x7f7f3c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:09.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.958+0000 7f7f3be00640 1 --2- 172.21.15.159:0/2292883561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f3c103e30 0x7f7f3c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:09.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.958+0000 7f7f3be00640 1 -- 172.21.15.159:0/2292883561 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3c19f490 con 0x7f7f3c108f60 2024-09-19T07:32:09.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.959+0000 7f7f3be00640 1 --2- 172.21.15.159:0/2292883561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f3c108f60 0x7f7f3c19edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009fd0 tx=0x7f7f3400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:09.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.959+0000 7f7f33e00640 1 -- 172.21.15.159:0/2292883561 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34017070 con 0x7f7f3c108f60 2024-09-19T07:32:09.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.959+0000 7f7f33e00640 1 -- 172.21.15.159:0/2292883561 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f34003e50 con 0x7f7f3c108f60 2024-09-19T07:32:09.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.959+0000 7f7f40e00640 1 -- 172.21.15.159:0/2292883561 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3c19f780 con 0x7f7f3c108f60 2024-09-19T07:32:09.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.959+0000 7f7f33e00640 1 -- 172.21.15.159:0/2292883561 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f340106c0 con 0x7f7f3c108f60 2024-09-19T07:32:09.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.959+0000 7f7f40e00640 1 -- 172.21.15.159:0/2292883561 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f3c1a7060 con 0x7f7f3c108f60 2024-09-19T07:32:09.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.960+0000 7f7f40e00640 1 -- 172.21.15.159:0/2292883561 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f3c105310 con 0x7f7f3c108f60 2024-09-19T07:32:09.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.962+0000 7f7f33e00640 1 -- 172.21.15.159:0/2292883561 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7f34003970 con 0x7f7f3c108f60 2024-09-19T07:32:09.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.963+0000 7f7f33e00640 1 --2- 172.21.15.159:0/2292883561 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f7f200a3350 0x7f7f200a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:09.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.963+0000 7f7f33e00640 1 -- 172.21.15.159:0/2292883561 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f7f340cfa50 con 0x7f7f3c108f60 2024-09-19T07:32:09.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.963+0000 7f7f3b400640 1 --2- 172.21.15.159:0/2292883561 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f7f200a3350 0x7f7f200a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:09.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.964+0000 7f7f3b400640 1 --2- 172.21.15.159:0/2292883561 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f7f200a3350 0x7f7f200a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009230 tx=0x7f7f2c0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:09.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:09.967+0000 7f7f33e00640 1 -- 172.21.15.159:0/2292883561 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7f3409bc00 con 0x7f7f3c108f60 2024-09-19T07:32:10.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:09 smithi203 ceph-mon[35335]: pgmap v347: 129 pgs: 129 active+clean; 629 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.8 KiB/s wr, 7 op/s 2024-09-19T07:32:10.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:09 smithi189 ceph-mon[34851]: pgmap v347: 129 pgs: 129 active+clean; 629 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.8 KiB/s wr, 7 op/s 2024-09-19T07:32:10.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:09 smithi159 ceph-mon[30472]: pgmap v347: 129 pgs: 129 active+clean; 629 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.8 KiB/s wr, 7 op/s 2024-09-19T07:32:10.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:10.146+0000 7f7f40e00640 1 -- 172.21.15.159:0/2292883561 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f3c10c170 con 0x7f7f3c108f60 2024-09-19T07:32:10.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:10.148+0000 7f7f33e00640 1 -- 172.21.15.159:0/2292883561 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24765 (secure 0 0 0) 0x7f7f3409ff00 con 0x7f7f3c108f60 2024-09-19T07:32:10.149 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:32:10.150 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":98,"fsid":"c9d18048-7657-11ef-bceb-c7b262605968","created":"2024-09-19T07:22:32.059836+0000","modified":"2024-09-19T07:31:31.799299+0000","last_up_change":"2024-09-19T07:30:03.328697+0000","last_in_change":"2024-09-19T07:29:38.987579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:25:55.202457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:30:58.022056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:31:01.042290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:31:10.977289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"752f5466-fe04-408a-8566-86625947fd3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6803","nonce":2738900830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6805","nonce":2738900830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6809","nonce":2738900830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6807","nonce":2738900830}]},"public_addr":"172.21.15.159:6803/2738900830","cluster_addr":"172.21.15.159:6805/2738900830","heartbeat_back_addr":"172.21.15.159:6809/2738900830","heartbeat_front_addr":"172.21.15.159:6807/2738900830","state":["exists","up"]},{"osd":1,"uuid":"7513adda-ca82-4df4-aa99-3b050b410424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6801","nonce":265696178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6803","nonce":265696178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6807","nonce":265696178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6805","nonce":265696178}]},"public_addr":"172.21.15.189:6801/265696178","cluster_addr":"172.21.15.189:6803/265696178","heartbeat_back_addr":"172.21.15.189:6807/265696178","heartbeat_front_addr":"172.21.15.189:6805/265696178","state":["exists","up"]},{"osd":2,"uuid":"aa03be79-58d8-4eb7-a051-0b46c502b12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6801","nonce":1826565288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6803","nonce":1826565288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6807","nonce":1826565288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6805","nonce":1826565288}]},"public_addr":"172.21.15.203:6801/1826565288","cluster_addr":"172.21.15.203:6803/1826565288","heartbeat_back_addr":"172.21.15.203:6807/1826565288","heartbeat_front_addr":"172.21.15.203:6805/1826565288","state":["exists","up"]},{"osd":3,"uuid":"fcd734f1-f40f-4566-82ab-c4607039f23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6811","nonce":109457380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6813","nonce":109457380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6817","nonce":109457380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6815","nonce":109457380}]},"public_addr":"172.21.15.159:6811/109457380","cluster_addr":"172.21.15.159:6813/109457380","heartbeat_back_addr":"172.21.15.159:6817/109457380","heartbeat_front_addr":"172.21.15.159:6815/109457380","state":["exists","up"]},{"osd":4,"uuid":"bc344c9e-2267-4058-bbe1-9c270327db8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6809","nonce":1036888180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6811","nonce":1036888180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6815","nonce":1036888180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6813","nonce":1036888180}]},"public_addr":"172.21.15.189:6809/1036888180","cluster_addr":"172.21.15.189:6811/1036888180","heartbeat_back_addr":"172.21.15.189:6815/1036888180","heartbeat_front_addr":"172.21.15.189:6813/1036888180","state":["exists","up"]},{"osd":5,"uuid":"86c48303-929d-42f4-bf48-cbb32649f95f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6809","nonce":4212601512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6811","nonce":4212601512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6815","nonce":4212601512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6813","nonce":4212601512}]},"public_addr":"172.21.15.203:6809/4212601512","cluster_addr":"172.21.15.203:6811/4212601512","heartbeat_back_addr":"172.21.15.203:6815/4212601512","heartbeat_front_addr":"172.21.15.203:6813/4212601512","state":["exists","up"]},{"osd":6,"uuid":"f0176ed1-c3a7-4d75-b213-fe88fab4441a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6819","nonce":3817684578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6821","nonce":3817684578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6825","nonce":3817684578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6823","nonce":3817684578}]},"public_addr":"172.21.15.159:6819/3817684578","cluster_addr":"172.21.15.159:6821/3817684578","heartbeat_back_addr":"172.21.15.159:6825/3817684578","heartbeat_front_addr":"172.21.15.159:6823/3817684578","state":["exists","up"]},{"osd":7,"uuid":"921bd59d-9415-4559-8e9f-035d336afd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6817","nonce":3936826456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6819","nonce":3936826456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6823","nonce":3936826456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6821","nonce":3936826456}]},"public_addr":"172.21.15.189:6817/3936826456","cluster_addr":"172.21.15.189:6819/3936826456","heartbeat_back_addr":"172.21.15.189:6823/3936826456","heartbeat_front_addr":"172.21.15.189:6821/3936826456","state":["exists","up"]},{"osd":8,"uuid":"ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6817","nonce":4062926645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6819","nonce":4062926645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6823","nonce":4062926645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6821","nonce":4062926645}]},"public_addr":"172.21.15.203:6817/4062926645","cluster_addr":"172.21.15.203:6819/4062926645","heartbeat_back_addr":"172.21.15.203:6823/4062926645","heartbeat_front_addr":"172.21.15.203:6821/4062926645","state":["exists","up"]},{"osd":9,"uuid":"df29a10f-1c60-41cb-9c2d-bc9f1cb174bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6827","nonce":4185848747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6829","nonce":4185848747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6833","nonce":4185848747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6831","nonce":4185848747}]},"public_addr":"172.21.15.159:6827/4185848747","cluster_addr":"172.21.15.159:6829/4185848747","heartbeat_back_addr":"172.21.15.159:6833/4185848747","heartbeat_front_addr":"172.21.15.159:6831/4185848747","state":["exists","up"]},{"osd":10,"uuid":"ccc440b4-4682-4998-bb53-95e24423cd1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6825","nonce":3018377605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6827","nonce":3018377605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6831","nonce":3018377605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6829","nonce":3018377605}]},"public_addr":"172.21.15.189:6825/3018377605","cluster_addr":"172.21.15.189:6827/3018377605","heartbeat_back_addr":"172.21.15.189:6831/3018377605","heartbeat_front_addr":"172.21.15.189:6829/3018377605","state":["exists","up"]},{"osd":11,"uuid":"e84afc5f-8051-4341-8fd0-3afb482ab4df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6825","nonce":2807503891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6827","nonce":2807503891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6831","nonce":2807503891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6829","nonce":2807503891}]},"public_addr":"172.21.15.203:6825/2807503891","cluster_addr":"172.21.15.203:6827/2807503891","heartbeat_back_addr":"172.21.15.203:6831/2807503891","heartbeat_front_addr":"172.21.15.203:6829/2807503891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:00.093677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:25.477205+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:51.345020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:21.077603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:48.822640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:13.516029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:44.382228+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:10.810700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:36.817155+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:07.728553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:34.747540+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:01.249332+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:0/2383222735":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1561371050":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6800/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:6800/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/2108730247":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/3393992615":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1366863724":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/1628344860":"2024-09-20T07:23:03.089190+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:32:10.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:10.150+0000 7f7f40e00640 1 -- 172.21.15.159:0/2292883561 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f7f200a3350 msgr2=0x7f7f200a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:10.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:10.150+0000 7f7f40e00640 1 --2- 172.21.15.159:0/2292883561 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f7f200a3350 0x7f7f200a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009230 tx=0x7f7f2c0058d0 comp rx=0 tx=0).stop 2024-09-19T07:32:10.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:10.150+0000 7f7f40e00640 1 -- 172.21.15.159:0/2292883561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f3c108f60 msgr2=0x7f7f3c19edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:10.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:10.150+0000 7f7f40e00640 1 --2- 172.21.15.159:0/2292883561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f3c108f60 0x7f7f3c19edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009fd0 tx=0x7f7f3400b260 comp rx=0 tx=0).stop 2024-09-19T07:32:10.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:10.150+0000 7f7f40e00640 1 -- 172.21.15.159:0/2292883561 shutdown_connections 2024-09-19T07:32:10.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:10.150+0000 7f7f40e00640 1 --2- 172.21.15.159:0/2292883561 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f7f200a3350 0x7f7f200a5810 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:10.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:10.150+0000 7f7f40e00640 1 --2- 172.21.15.159:0/2292883561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f3c108f60 0x7f7f3c19edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:10.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:10.150+0000 7f7f40e00640 1 --2- 172.21.15.159:0/2292883561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f3c1047e0 0x7f7f3c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:10.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:10.150+0000 7f7f40e00640 1 --2- 172.21.15.159:0/2292883561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f3c103e30 0x7f7f3c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:10.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:10.150+0000 7f7f40e00640 1 -- 172.21.15.159:0/2292883561 >> 172.21.15.159:0/2292883561 conn(0x7f7f3c0ff910 msgr2=0x7f7f3c100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:10.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:10.151+0000 7f7f40e00640 1 -- 172.21.15.159:0/2292883561 shutdown_connections 2024-09-19T07:32:10.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:10.151+0000 7f7f40e00640 1 -- 172.21.15.159:0/2292883561 wait complete. 2024-09-19T07:32:11.027 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:10 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2292883561' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:32:11.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:10 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/2292883561' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:32:11.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:10 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/2292883561' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:32:11.067 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T07:32:11.067 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T07:32:11.438 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:11.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:11 smithi159 ceph-mon[30472]: pgmap v348: 129 pgs: 129 active+clean; 629 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.9 KiB/s wr, 7 op/s 2024-09-19T07:32:12.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.024+0000 7ff3d1a00640 1 -- 172.21.15.159:0/1889809477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3cc101de0 msgr2=0x7ff3cc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:12.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.024+0000 7ff3d1a00640 1 --2- 172.21.15.159:0/1889809477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3cc101de0 0x7ff3cc1021c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009900 tx=0x7ff3bc02a3f0 comp rx=0 tx=0).stop 2024-09-19T07:32:12.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.024+0000 7ff3d1a00640 1 -- 172.21.15.159:0/1889809477 shutdown_connections 2024-09-19T07:32:12.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.024+0000 7ff3d1a00640 1 --2- 172.21.15.159:0/1889809477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3cc106f10 0x7ff3cc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:12.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.024+0000 7ff3d1a00640 1 --2- 172.21.15.159:0/1889809477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3cc102790 0x7ff3cc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:12.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.024+0000 7ff3d1a00640 1 --2- 172.21.15.159:0/1889809477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3cc101de0 0x7ff3cc1021c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:12.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.024+0000 7ff3d1a00640 1 -- 172.21.15.159:0/1889809477 >> 172.21.15.159:0/1889809477 conn(0x7ff3cc0fd8c0 msgr2=0x7ff3cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:12.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.024+0000 7ff3d1a00640 1 -- 172.21.15.159:0/1889809477 shutdown_connections 2024-09-19T07:32:12.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.024+0000 7ff3d1a00640 1 -- 172.21.15.159:0/1889809477 wait complete. 2024-09-19T07:32:12.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.025+0000 7ff3d1a00640 1 Processor -- start 2024-09-19T07:32:12.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.025+0000 7ff3d1a00640 1 -- start start 2024-09-19T07:32:12.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.026+0000 7ff3d1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3cc101de0 0x7ff3cc198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:12.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.026+0000 7ff3d1a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3cc102790 0x7ff3cc198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:12.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.026+0000 7ff3d1a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3cc106f10 0x7ff3cc19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:12.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.026+0000 7ff3d1a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc111fc0 con 0x7ff3cc101de0 2024-09-19T07:32:12.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.026+0000 7ff3d1a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc111e40 con 0x7ff3cc102790 2024-09-19T07:32:12.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.026+0000 7ff3d1a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc112140 con 0x7ff3cc106f10 2024-09-19T07:32:12.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.026+0000 7ff3caa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3cc101de0 0x7ff3cc198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:12.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.026+0000 7ff3caa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3cc101de0 0x7ff3cc198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38606/0 (socket says 172.21.15.159:38606) 2024-09-19T07:32:12.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.026+0000 7ff3caa00640 1 -- 172.21.15.159:0/2672761573 learned_addr learned my addr 172.21.15.159:0/2672761573 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:12.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.026+0000 7ff3ca000640 1 --2- 172.21.15.159:0/2672761573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3cc102790 0x7ff3cc198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:12.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.026+0000 7ff3cb400640 1 --2- 172.21.15.159:0/2672761573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3cc106f10 0x7ff3cc19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:12.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.027+0000 7ff3ca000640 1 -- 172.21.15.159:0/2672761573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3cc106f10 msgr2=0x7ff3cc19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:12.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.027+0000 7ff3ca000640 1 --2- 172.21.15.159:0/2672761573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3cc106f10 0x7ff3cc19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:12.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.027+0000 7ff3ca000640 1 -- 172.21.15.159:0/2672761573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3cc101de0 msgr2=0x7ff3cc198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:12.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.027+0000 7ff3ca000640 1 --2- 172.21.15.159:0/2672761573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3cc101de0 0x7ff3cc198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:12.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.027+0000 7ff3ca000640 1 -- 172.21.15.159:0/2672761573 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3cc19d510 con 0x7ff3cc102790 2024-09-19T07:32:12.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.027+0000 7ff3caa00640 1 --2- 172.21.15.159:0/2672761573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3cc101de0 0x7ff3cc198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:32:12.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.027+0000 7ff3ca000640 1 --2- 172.21.15.159:0/2672761573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3cc102790 0x7ff3cc198aa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800e9f0 tx=0x7ff3b800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:12.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.027+0000 7ff3c3400640 1 -- 172.21.15.159:0/2672761573 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b800ccc0 con 0x7ff3cc102790 2024-09-19T07:32:12.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.027+0000 7ff3d1a00640 1 -- 172.21.15.159:0/2672761573 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3cc19d800 con 0x7ff3cc102790 2024-09-19T07:32:12.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.027+0000 7ff3c3400640 1 -- 172.21.15.159:0/2672761573 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b8005020 con 0x7ff3cc102790 2024-09-19T07:32:12.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.027+0000 7ff3c3400640 1 -- 172.21.15.159:0/2672761573 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8004a00 con 0x7ff3cc102790 2024-09-19T07:32:12.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.028+0000 7ff3d1a00640 1 -- 172.21.15.159:0/2672761573 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3cc1a50e0 con 0x7ff3cc102790 2024-09-19T07:32:12.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.029+0000 7ff3d1a00640 1 -- 172.21.15.159:0/2672761573 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3cc1021c0 con 0x7ff3cc102790 2024-09-19T07:32:12.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.031+0000 7ff3c3400640 1 -- 172.21.15.159:0/2672761573 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff3b80051d0 con 0x7ff3cc102790 2024-09-19T07:32:12.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.032+0000 7ff3c3400640 1 --2- 172.21.15.159:0/2672761573 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff3b00a3350 0x7ff3b00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:12.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.033+0000 7ff3c3400640 1 -- 172.21.15.159:0/2672761573 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7ff3b80cffc0 con 0x7ff3cc102790 2024-09-19T07:32:12.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.033+0000 7ff3caa00640 1 --2- 172.21.15.159:0/2672761573 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff3b00a3350 0x7ff3b00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:12.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.033+0000 7ff3caa00640 1 --2- 172.21.15.159:0/2672761573 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff3b00a3350 0x7ff3b00a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0096c0 tx=0x7ff3bc0046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:12.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.034+0000 7ff3c3400640 1 -- 172.21.15.159:0/2672761573 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff3b809c7c0 con 0x7ff3cc102790 2024-09-19T07:32:12.035 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:11 smithi203 ceph-mon[35335]: pgmap v348: 129 pgs: 129 active+clean; 629 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.9 KiB/s wr, 7 op/s 2024-09-19T07:32:12.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:11 smithi189 ceph-mon[34851]: pgmap v348: 129 pgs: 129 active+clean; 629 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.9 KiB/s wr, 7 op/s 2024-09-19T07:32:12.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.206+0000 7ff3d1a00640 1 -- 172.21.15.159:0/2672761573 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3cc10a120 con 0x7ff3b00a3350 2024-09-19T07:32:12.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.218+0000 7ff3c3400640 1 -- 172.21.15.159:0/2672761573 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340649 (secure 0 0 0) 0x7ff3cc10a120 con 0x7ff3b00a3350 2024-09-19T07:32:12.223 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:32:12.225 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-19T07:32:12.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.224+0000 7ff3d1a00640 1 -- 172.21.15.159:0/2672761573 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff3b00a3350 msgr2=0x7ff3b00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:12.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.224+0000 7ff3d1a00640 1 --2- 172.21.15.159:0/2672761573 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff3b00a3350 0x7ff3b00a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0096c0 tx=0x7ff3bc0046c0 comp rx=0 tx=0).stop 2024-09-19T07:32:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.224+0000 7ff3d1a00640 1 -- 172.21.15.159:0/2672761573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3cc102790 msgr2=0x7ff3cc198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.224+0000 7ff3d1a00640 1 --2- 172.21.15.159:0/2672761573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3cc102790 0x7ff3cc198aa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800e9f0 tx=0x7ff3b800edf0 comp rx=0 tx=0).stop 2024-09-19T07:32:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.224+0000 7ff3d1a00640 1 -- 172.21.15.159:0/2672761573 shutdown_connections 2024-09-19T07:32:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.224+0000 7ff3d1a00640 1 --2- 172.21.15.159:0/2672761573 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff3b00a3350 0x7ff3b00a5810 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.224+0000 7ff3d1a00640 1 --2- 172.21.15.159:0/2672761573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3cc106f10 0x7ff3cc19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.224+0000 7ff3d1a00640 1 --2- 172.21.15.159:0/2672761573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3cc102790 0x7ff3cc198aa0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.224+0000 7ff3d1a00640 1 --2- 172.21.15.159:0/2672761573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3cc101de0 0x7ff3cc198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.224+0000 7ff3d1a00640 1 -- 172.21.15.159:0/2672761573 >> 172.21.15.159:0/2672761573 conn(0x7ff3cc0fd8c0 msgr2=0x7ff3cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.224+0000 7ff3d1a00640 1 -- 172.21.15.159:0/2672761573 shutdown_connections 2024-09-19T07:32:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:12.224+0000 7ff3d1a00640 1 -- 172.21.15.159:0/2672761573 wait complete. 2024-09-19T07:32:13.185 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-09-19T07:32:10.706948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644096,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":238,"num_read_kb":223,"num_write":339,"num_write_kb":3040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353992,"kb_used_data":13064,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124587832,"statfs":{"total":1151940427776,"available":1151577939968,"internally_reserved":0,"allocated":13377536,"data_stored":8546582,"data_compressed":430272,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46274,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":77,"num_write":29,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003351"},"pg_stats":[{"pgid":"2.1f","version":"98'9","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:50.896695+0000","last_change":"2024-09-19T07:30:59.408103+0000","last_active":"2024-09-19T07:31:50.896695+0000","last_peered":"2024-09-19T07:31:50.896695+0000","last_clean":"2024-09-19T07:31:50.896695+0000","last_became_active":"2024-09-19T07:30:59.407585+0000","last_became_peered":"2024-09-19T07:30:59.407585+0000","last_unstale":"2024-09-19T07:31:50.896695+0000","last_undegraded":"2024-09-19T07:31:50.896695+0000","last_fullsized":"2024-09-19T07:31:50.896695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:03.104939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806455+0000","last_change":"2024-09-19T07:31:02.438518+0000","last_active":"2024-09-19T07:31:31.806455+0000","last_peered":"2024-09-19T07:31:31.806455+0000","last_clean":"2024-09-19T07:31:31.806455+0000","last_became_active":"2024-09-19T07:31:02.438025+0000","last_became_peered":"2024-09-19T07:31:02.438025+0000","last_unstale":"2024-09-19T07:31:31.806455+0000","last_undegraded":"2024-09-19T07:31:31.806455+0000","last_fullsized":"2024-09-19T07:31:31.806455+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:59.148172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803301+0000","last_change":"2024-09-19T07:31:12.503517+0000","last_active":"2024-09-19T07:31:31.803301+0000","last_peered":"2024-09-19T07:31:31.803301+0000","last_clean":"2024-09-19T07:31:31.803301+0000","last_became_active":"2024-09-19T07:31:12.503155+0000","last_became_peered":"2024-09-19T07:31:12.503155+0000","last_unstale":"2024-09-19T07:31:31.803301+0000","last_undegraded":"2024-09-19T07:31:31.803301+0000","last_fullsized":"2024-09-19T07:31:31.803301+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:15.475388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806651+0000","last_change":"2024-09-19T07:30:59.445107+0000","last_active":"2024-09-19T07:31:31.806651+0000","last_peered":"2024-09-19T07:31:31.806651+0000","last_clean":"2024-09-19T07:31:31.806651+0000","last_became_active":"2024-09-19T07:30:59.444782+0000","last_became_peered":"2024-09-19T07:30:59.444782+0000","last_unstale":"2024-09-19T07:31:31.806651+0000","last_undegraded":"2024-09-19T07:31:31.806651+0000","last_fullsized":"2024-09-19T07:31:31.806651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:37.582314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.020219+0000","last_change":"2024-09-19T07:31:02.437690+0000","last_active":"2024-09-19T07:31:32.020219+0000","last_peered":"2024-09-19T07:31:32.020219+0000","last_clean":"2024-09-19T07:31:32.020219+0000","last_became_active":"2024-09-19T07:31:02.429969+0000","last_became_peered":"2024-09-19T07:31:02.429969+0000","last_unstale":"2024-09-19T07:31:32.020219+0000","last_undegraded":"2024-09-19T07:31:32.020219+0000","last_fullsized":"2024-09-19T07:31:32.020219+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:23.227114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805966+0000","last_change":"2024-09-19T07:31:12.500314+0000","last_active":"2024-09-19T07:31:31.805966+0000","last_peered":"2024-09-19T07:31:31.805966+0000","last_clean":"2024-09-19T07:31:31.805966+0000","last_became_active":"2024-09-19T07:31:12.499529+0000","last_became_peered":"2024-09-19T07:31:12.499529+0000","last_unstale":"2024-09-19T07:31:31.805966+0000","last_undegraded":"2024-09-19T07:31:31.805966+0000","last_fullsized":"2024-09-19T07:31:31.805966+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:46.805336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:50.895345+0000","last_change":"2024-09-19T07:30:59.408974+0000","last_active":"2024-09-19T07:31:50.895345+0000","last_peered":"2024-09-19T07:31:50.895345+0000","last_clean":"2024-09-19T07:31:50.895345+0000","last_became_active":"2024-09-19T07:30:59.408606+0000","last_became_peered":"2024-09-19T07:30:59.408606+0000","last_unstale":"2024-09-19T07:31:50.895345+0000","last_undegraded":"2024-09-19T07:31:50.895345+0000","last_fullsized":"2024-09-19T07:31:50.895345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:21.962123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.245608+0000","last_change":"2024-09-19T07:31:02.431202+0000","last_active":"2024-09-19T07:31:32.245608+0000","last_peered":"2024-09-19T07:31:32.245608+0000","last_clean":"2024-09-19T07:31:32.245608+0000","last_became_active":"2024-09-19T07:31:02.430789+0000","last_became_peered":"2024-09-19T07:31:02.430789+0000","last_unstale":"2024-09-19T07:31:32.245608+0000","last_undegraded":"2024-09-19T07:31:32.245608+0000","last_fullsized":"2024-09-19T07:31:32.245608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:40.750108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.807909+0000","last_change":"2024-09-19T07:31:12.500000+0000","last_active":"2024-09-19T07:31:31.807909+0000","last_peered":"2024-09-19T07:31:31.807909+0000","last_clean":"2024-09-19T07:31:31.807909+0000","last_became_active":"2024-09-19T07:31:12.499526+0000","last_became_peered":"2024-09-19T07:31:12.499526+0000","last_unstale":"2024-09-19T07:31:31.807909+0000","last_undegraded":"2024-09-19T07:31:31.807909+0000","last_fullsized":"2024-09-19T07:31:31.807909+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:15.449391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.246964+0000","last_change":"2024-09-19T07:30:59.411645+0000","last_active":"2024-09-19T07:31:32.246964+0000","last_peered":"2024-09-19T07:31:32.246964+0000","last_clean":"2024-09-19T07:31:32.246964+0000","last_became_active":"2024-09-19T07:30:59.411262+0000","last_became_peered":"2024-09-19T07:30:59.411262+0000","last_unstale":"2024-09-19T07:31:32.246964+0000","last_undegraded":"2024-09-19T07:31:32.246964+0000","last_fullsized":"2024-09-19T07:31:32.246964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:53.538203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.247958+0000","last_change":"2024-09-19T07:31:32.247958+0000","last_active":"2024-09-19T07:31:32.247958+0000","last_peered":"2024-09-19T07:31:32.247958+0000","last_clean":"2024-09-19T07:31:32.247958+0000","last_became_active":"2024-09-19T07:31:32.247521+0000","last_became_peered":"2024-09-19T07:31:32.247521+0000","last_unstale":"2024-09-19T07:31:32.247958+0000","last_undegraded":"2024-09-19T07:31:32.247958+0000","last_fullsized":"2024-09-19T07:31:32.247958+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:38.291915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806311+0000","last_change":"2024-09-19T07:31:12.500070+0000","last_active":"2024-09-19T07:31:31.806311+0000","last_peered":"2024-09-19T07:31:31.806311+0000","last_clean":"2024-09-19T07:31:31.806311+0000","last_became_active":"2024-09-19T07:31:12.499658+0000","last_became_peered":"2024-09-19T07:31:12.499658+0000","last_unstale":"2024-09-19T07:31:31.806311+0000","last_undegraded":"2024-09-19T07:31:31.806311+0000","last_fullsized":"2024-09-19T07:31:31.806311+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:08.955749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531082+0000","last_change":"2024-09-19T07:31:02.426972+0000","last_active":"2024-09-19T07:31:32.531082+0000","last_peered":"2024-09-19T07:31:32.531082+0000","last_clean":"2024-09-19T07:31:32.531082+0000","last_became_active":"2024-09-19T07:31:02.426661+0000","last_became_peered":"2024-09-19T07:31:02.426661+0000","last_unstale":"2024-09-19T07:31:32.531082+0000","last_undegraded":"2024-09-19T07:31:32.531082+0000","last_fullsized":"2024-09-19T07:31:32.531082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:26.943324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:49.886419+0000","last_change":"2024-09-19T07:30:59.410056+0000","last_active":"2024-09-19T07:31:49.886419+0000","last_peered":"2024-09-19T07:31:49.886419+0000","last_clean":"2024-09-19T07:31:49.886419+0000","last_became_active":"2024-09-19T07:30:59.409669+0000","last_became_peered":"2024-09-19T07:30:59.409669+0000","last_unstale":"2024-09-19T07:31:49.886419+0000","last_undegraded":"2024-09-19T07:31:49.886419+0000","last_fullsized":"2024-09-19T07:31:49.886419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:43.169905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806430+0000","last_change":"2024-09-19T07:31:12.557969+0000","last_active":"2024-09-19T07:31:31.806430+0000","last_peered":"2024-09-19T07:31:31.806430+0000","last_clean":"2024-09-19T07:31:31.806430+0000","last_became_active":"2024-09-19T07:31:12.556692+0000","last_became_peered":"2024-09-19T07:31:12.556692+0000","last_unstale":"2024-09-19T07:31:31.806430+0000","last_undegraded":"2024-09-19T07:31:31.806430+0000","last_fullsized":"2024-09-19T07:31:31.806430+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.563341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806099+0000","last_change":"2024-09-19T07:31:02.438979+0000","last_active":"2024-09-19T07:31:31.806099+0000","last_peered":"2024-09-19T07:31:31.806099+0000","last_clean":"2024-09-19T07:31:31.806099+0000","last_became_active":"2024-09-19T07:31:02.437192+0000","last_became_peered":"2024-09-19T07:31:02.437192+0000","last_unstale":"2024-09-19T07:31:31.806099+0000","last_undegraded":"2024-09-19T07:31:31.806099+0000","last_fullsized":"2024-09-19T07:31:31.806099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:39.957366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803751+0000","last_change":"2024-09-19T07:30:59.406970+0000","last_active":"2024-09-19T07:31:31.803751+0000","last_peered":"2024-09-19T07:31:31.803751+0000","last_clean":"2024-09-19T07:31:31.803751+0000","last_became_active":"2024-09-19T07:30:59.406537+0000","last_became_peered":"2024-09-19T07:30:59.406537+0000","last_unstale":"2024-09-19T07:31:31.803751+0000","last_undegraded":"2024-09-19T07:31:31.803751+0000","last_fullsized":"2024-09-19T07:31:31.803751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:32.969944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806387+0000","last_change":"2024-09-19T07:31:12.495709+0000","last_active":"2024-09-19T07:31:31.806387+0000","last_peered":"2024-09-19T07:31:31.806387+0000","last_clean":"2024-09-19T07:31:31.806387+0000","last_became_active":"2024-09-19T07:31:12.495201+0000","last_became_peered":"2024-09-19T07:31:12.495201+0000","last_unstale":"2024-09-19T07:31:31.806387+0000","last_undegraded":"2024-09-19T07:31:31.806387+0000","last_fullsized":"2024-09-19T07:31:31.806387+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:48.710011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.804813+0000","last_change":"2024-09-19T07:31:02.431032+0000","last_active":"2024-09-19T07:31:31.804813+0000","last_peered":"2024-09-19T07:31:31.804813+0000","last_clean":"2024-09-19T07:31:31.804813+0000","last_became_active":"2024-09-19T07:31:02.430691+0000","last_became_peered":"2024-09-19T07:31:02.430691+0000","last_unstale":"2024-09-19T07:31:31.804813+0000","last_undegraded":"2024-09-19T07:31:31.804813+0000","last_fullsized":"2024-09-19T07:31:31.804813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:59.627125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:49.887107+0000","last_change":"2024-09-19T07:30:59.410931+0000","last_active":"2024-09-19T07:31:49.887107+0000","last_peered":"2024-09-19T07:31:49.887107+0000","last_clean":"2024-09-19T07:31:49.887107+0000","last_became_active":"2024-09-19T07:30:59.410516+0000","last_became_peered":"2024-09-19T07:30:59.410516+0000","last_unstale":"2024-09-19T07:31:49.887107+0000","last_undegraded":"2024-09-19T07:31:49.887107+0000","last_fullsized":"2024-09-19T07:31:49.887107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:39.688441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.809669+0000","last_change":"2024-09-19T07:31:31.809224+0000","last_active":"2024-09-19T07:31:31.809669+0000","last_peered":"2024-09-19T07:31:31.809669+0000","last_clean":"2024-09-19T07:31:31.809669+0000","last_became_active":"2024-09-19T07:31:31.808673+0000","last_became_peered":"2024-09-19T07:31:31.808673+0000","last_unstale":"2024-09-19T07:31:31.809669+0000","last_undegraded":"2024-09-19T07:31:31.809669+0000","last_fullsized":"2024-09-19T07:31:31.809669+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:38.675403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.246222+0000","last_change":"2024-09-19T07:31:02.426735+0000","last_active":"2024-09-19T07:31:32.246222+0000","last_peered":"2024-09-19T07:31:32.246222+0000","last_clean":"2024-09-19T07:31:32.246222+0000","last_became_active":"2024-09-19T07:31:02.426324+0000","last_became_peered":"2024-09-19T07:31:02.426324+0000","last_unstale":"2024-09-19T07:31:32.246222+0000","last_undegraded":"2024-09-19T07:31:32.246222+0000","last_fullsized":"2024-09-19T07:31:32.246222+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:10.171191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:49.883039+0000","last_change":"2024-09-19T07:30:59.445125+0000","last_active":"2024-09-19T07:31:49.883039+0000","last_peered":"2024-09-19T07:31:49.883039+0000","last_clean":"2024-09-19T07:31:49.883039+0000","last_became_active":"2024-09-19T07:30:59.444757+0000","last_became_peered":"2024-09-19T07:30:59.444757+0000","last_unstale":"2024-09-19T07:31:49.883039+0000","last_undegraded":"2024-09-19T07:31:49.883039+0000","last_fullsized":"2024-09-19T07:31:49.883039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:34.183389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.807249+0000","last_change":"2024-09-19T07:31:12.557277+0000","last_active":"2024-09-19T07:31:31.807249+0000","last_peered":"2024-09-19T07:31:31.807249+0000","last_clean":"2024-09-19T07:31:31.807249+0000","last_became_active":"2024-09-19T07:31:12.556885+0000","last_became_peered":"2024-09-19T07:31:12.556885+0000","last_unstale":"2024-09-19T07:31:31.807249+0000","last_undegraded":"2024-09-19T07:31:31.807249+0000","last_fullsized":"2024-09-19T07:31:31.807249+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:39.960404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.020126+0000","last_change":"2024-09-19T07:31:02.441712+0000","last_active":"2024-09-19T07:31:32.020126+0000","last_peered":"2024-09-19T07:31:32.020126+0000","last_clean":"2024-09-19T07:31:32.020126+0000","last_became_active":"2024-09-19T07:31:02.440039+0000","last_became_peered":"2024-09-19T07:31:02.440039+0000","last_unstale":"2024-09-19T07:31:32.020126+0000","last_undegraded":"2024-09-19T07:31:32.020126+0000","last_fullsized":"2024-09-19T07:31:32.020126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:13.601774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:49.886162+0000","last_change":"2024-09-19T07:31:00.027265+0000","last_active":"2024-09-19T07:31:49.886162+0000","last_peered":"2024-09-19T07:31:49.886162+0000","last_clean":"2024-09-19T07:31:49.886162+0000","last_became_active":"2024-09-19T07:31:00.026924+0000","last_became_peered":"2024-09-19T07:31:00.026924+0000","last_unstale":"2024-09-19T07:31:49.886162+0000","last_undegraded":"2024-09-19T07:31:49.886162+0000","last_fullsized":"2024-09-19T07:31:49.886162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:05.290978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803449+0000","last_change":"2024-09-19T07:31:12.495613+0000","last_active":"2024-09-19T07:31:31.803449+0000","last_peered":"2024-09-19T07:31:31.803449+0000","last_clean":"2024-09-19T07:31:31.803449+0000","last_became_active":"2024-09-19T07:31:12.495218+0000","last_became_peered":"2024-09-19T07:31:12.495218+0000","last_unstale":"2024-09-19T07:31:31.803449+0000","last_undegraded":"2024-09-19T07:31:31.803449+0000","last_fullsized":"2024-09-19T07:31:31.803449+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:27.651671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806435+0000","last_change":"2024-09-19T07:31:02.442902+0000","last_active":"2024-09-19T07:31:31.806435+0000","last_peered":"2024-09-19T07:31:31.806435+0000","last_clean":"2024-09-19T07:31:31.806435+0000","last_became_active":"2024-09-19T07:31:02.437173+0000","last_became_peered":"2024-09-19T07:31:02.437173+0000","last_unstale":"2024-09-19T07:31:31.806435+0000","last_undegraded":"2024-09-19T07:31:31.806435+0000","last_fullsized":"2024-09-19T07:31:31.806435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:40:44.760641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'8","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:32:05.443697+0000","last_change":"2024-09-19T07:30:59.406488+0000","last_active":"2024-09-19T07:32:05.443697+0000","last_peered":"2024-09-19T07:32:05.443697+0000","last_clean":"2024-09-19T07:32:05.443697+0000","last_became_active":"2024-09-19T07:30:59.406121+0000","last_became_peered":"2024-09-19T07:30:59.406121+0000","last_unstale":"2024-09-19T07:32:05.443697+0000","last_undegraded":"2024-09-19T07:32:05.443697+0000","last_fullsized":"2024-09-19T07:32:05.443697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:14:50.534009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805273+0000","last_change":"2024-09-19T07:31:12.558127+0000","last_active":"2024-09-19T07:31:31.805273+0000","last_peered":"2024-09-19T07:31:31.805273+0000","last_clean":"2024-09-19T07:31:31.805273+0000","last_became_active":"2024-09-19T07:31:12.557078+0000","last_became_peered":"2024-09-19T07:31:12.557078+0000","last_unstale":"2024-09-19T07:31:31.805273+0000","last_undegraded":"2024-09-19T07:31:31.805273+0000","last_fullsized":"2024-09-19T07:31:31.805273+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:23.076116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803172+0000","last_change":"2024-09-19T07:31:02.433918+0000","last_active":"2024-09-19T07:31:31.803172+0000","last_peered":"2024-09-19T07:31:31.803172+0000","last_clean":"2024-09-19T07:31:31.803172+0000","last_became_active":"2024-09-19T07:31:02.430802+0000","last_became_peered":"2024-09-19T07:31:02.430802+0000","last_unstale":"2024-09-19T07:31:31.803172+0000","last_undegraded":"2024-09-19T07:31:31.803172+0000","last_fullsized":"2024-09-19T07:31:31.803172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:28.948441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:49.886123+0000","last_change":"2024-09-19T07:30:59.444178+0000","last_active":"2024-09-19T07:31:49.886123+0000","last_peered":"2024-09-19T07:31:49.886123+0000","last_clean":"2024-09-19T07:31:49.886123+0000","last_became_active":"2024-09-19T07:30:59.443847+0000","last_became_peered":"2024-09-19T07:30:59.443847+0000","last_unstale":"2024-09-19T07:31:49.886123+0000","last_undegraded":"2024-09-19T07:31:49.886123+0000","last_fullsized":"2024-09-19T07:31:49.886123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:08.588244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805143+0000","last_change":"2024-09-19T07:31:12.508847+0000","last_active":"2024-09-19T07:31:31.805143+0000","last_peered":"2024-09-19T07:31:31.805143+0000","last_clean":"2024-09-19T07:31:31.805143+0000","last_became_active":"2024-09-19T07:31:12.508371+0000","last_became_peered":"2024-09-19T07:31:12.508371+0000","last_unstale":"2024-09-19T07:31:31.805143+0000","last_undegraded":"2024-09-19T07:31:31.805143+0000","last_fullsized":"2024-09-19T07:31:31.805143+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:27.420724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805784+0000","last_change":"2024-09-19T07:31:02.438213+0000","last_active":"2024-09-19T07:31:31.805784+0000","last_peered":"2024-09-19T07:31:31.805784+0000","last_clean":"2024-09-19T07:31:31.805784+0000","last_became_active":"2024-09-19T07:31:02.437892+0000","last_became_peered":"2024-09-19T07:31:02.437892+0000","last_unstale":"2024-09-19T07:31:31.805784+0000","last_undegraded":"2024-09-19T07:31:31.805784+0000","last_fullsized":"2024-09-19T07:31:31.805784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:50.249962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.809028+0000","last_change":"2024-09-19T07:30:59.445523+0000","last_active":"2024-09-19T07:31:31.809028+0000","last_peered":"2024-09-19T07:31:31.809028+0000","last_clean":"2024-09-19T07:31:31.809028+0000","last_became_active":"2024-09-19T07:30:59.445118+0000","last_became_peered":"2024-09-19T07:30:59.445118+0000","last_unstale":"2024-09-19T07:31:31.809028+0000","last_undegraded":"2024-09-19T07:31:31.809028+0000","last_fullsized":"2024-09-19T07:31:31.809028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:35.369102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.246349+0000","last_change":"2024-09-19T07:31:12.557541+0000","last_active":"2024-09-19T07:31:32.246349+0000","last_peered":"2024-09-19T07:31:32.246349+0000","last_clean":"2024-09-19T07:31:32.246349+0000","last_became_active":"2024-09-19T07:31:12.557130+0000","last_became_peered":"2024-09-19T07:31:12.557130+0000","last_unstale":"2024-09-19T07:31:32.246349+0000","last_undegraded":"2024-09-19T07:31:32.246349+0000","last_fullsized":"2024-09-19T07:31:32.246349+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:52.610222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805541+0000","last_change":"2024-09-19T07:31:02.434743+0000","last_active":"2024-09-19T07:31:31.805541+0000","last_peered":"2024-09-19T07:31:31.805541+0000","last_clean":"2024-09-19T07:31:31.805541+0000","last_became_active":"2024-09-19T07:31:02.434312+0000","last_became_peered":"2024-09-19T07:31:02.434312+0000","last_unstale":"2024-09-19T07:31:31.805541+0000","last_undegraded":"2024-09-19T07:31:31.805541+0000","last_fullsized":"2024-09-19T07:31:31.805541+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:24:00.569195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531239+0000","last_change":"2024-09-19T07:30:59.409893+0000","last_active":"2024-09-19T07:31:32.531239+0000","last_peered":"2024-09-19T07:31:32.531239+0000","last_clean":"2024-09-19T07:31:32.531239+0000","last_became_active":"2024-09-19T07:30:59.409440+0000","last_became_peered":"2024-09-19T07:30:59.409440+0000","last_unstale":"2024-09-19T07:31:32.531239+0000","last_undegraded":"2024-09-19T07:31:32.531239+0000","last_fullsized":"2024-09-19T07:31:32.531239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:45.999596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803030+0000","last_change":"2024-09-19T07:31:12.505965+0000","last_active":"2024-09-19T07:31:31.803030+0000","last_peered":"2024-09-19T07:31:31.803030+0000","last_clean":"2024-09-19T07:31:31.803030+0000","last_became_active":"2024-09-19T07:31:12.504675+0000","last_became_peered":"2024-09-19T07:31:12.504675+0000","last_unstale":"2024-09-19T07:31:31.803030+0000","last_undegraded":"2024-09-19T07:31:31.803030+0000","last_fullsized":"2024-09-19T07:31:31.803030+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:47.345020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.809488+0000","last_change":"2024-09-19T07:31:31.809488+0000","last_active":"2024-09-19T07:31:31.809488+0000","last_peered":"2024-09-19T07:31:31.809488+0000","last_clean":"2024-09-19T07:31:31.809488+0000","last_became_active":"2024-09-19T07:31:31.809107+0000","last_became_peered":"2024-09-19T07:31:31.809107+0000","last_unstale":"2024-09-19T07:31:31.809488+0000","last_undegraded":"2024-09-19T07:31:31.809488+0000","last_fullsized":"2024-09-19T07:31:31.809488+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:39.757841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":60,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:32:05.917404+0000","last_change":"2024-09-19T07:30:59.445001+0000","last_active":"2024-09-19T07:32:05.917404+0000","last_peered":"2024-09-19T07:32:05.917404+0000","last_clean":"2024-09-19T07:32:05.917404+0000","last_became_active":"2024-09-19T07:30:59.444590+0000","last_became_peered":"2024-09-19T07:30:59.444590+0000","last_unstale":"2024-09-19T07:32:05.917404+0000","last_undegraded":"2024-09-19T07:32:05.917404+0000","last_fullsized":"2024-09-19T07:32:05.917404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:13.934085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803248+0000","last_change":"2024-09-19T07:31:12.557622+0000","last_active":"2024-09-19T07:31:31.803248+0000","last_peered":"2024-09-19T07:31:31.803248+0000","last_clean":"2024-09-19T07:31:31.803248+0000","last_became_active":"2024-09-19T07:31:12.556934+0000","last_became_peered":"2024-09-19T07:31:12.556934+0000","last_unstale":"2024-09-19T07:31:31.803248+0000","last_undegraded":"2024-09-19T07:31:31.803248+0000","last_fullsized":"2024-09-19T07:31:31.803248+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:06:31.944819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.802694+0000","last_change":"2024-09-19T07:31:02.433199+0000","last_active":"2024-09-19T07:31:31.802694+0000","last_peered":"2024-09-19T07:31:31.802694+0000","last_clean":"2024-09-19T07:31:31.802694+0000","last_became_active":"2024-09-19T07:31:02.432739+0000","last_became_peered":"2024-09-19T07:31:02.432739+0000","last_unstale":"2024-09-19T07:31:31.802694+0000","last_undegraded":"2024-09-19T07:31:31.802694+0000","last_fullsized":"2024-09-19T07:31:31.802694+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:57.392885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:50.894688+0000","last_change":"2024-09-19T07:30:59.413041+0000","last_active":"2024-09-19T07:31:50.894688+0000","last_peered":"2024-09-19T07:31:50.894688+0000","last_clean":"2024-09-19T07:31:50.894688+0000","last_became_active":"2024-09-19T07:30:59.412483+0000","last_became_peered":"2024-09-19T07:30:59.412483+0000","last_unstale":"2024-09-19T07:31:50.894688+0000","last_undegraded":"2024-09-19T07:31:50.894688+0000","last_fullsized":"2024-09-19T07:31:50.894688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:05.151006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806998+0000","last_change":"2024-09-19T07:31:12.557638+0000","last_active":"2024-09-19T07:31:31.806998+0000","last_peered":"2024-09-19T07:31:31.806998+0000","last_clean":"2024-09-19T07:31:31.806998+0000","last_became_active":"2024-09-19T07:31:12.557170+0000","last_became_peered":"2024-09-19T07:31:12.557170+0000","last_unstale":"2024-09-19T07:31:31.806998+0000","last_undegraded":"2024-09-19T07:31:31.806998+0000","last_fullsized":"2024-09-19T07:31:31.806998+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:14.077038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806268+0000","last_change":"2024-09-19T07:31:02.443081+0000","last_active":"2024-09-19T07:31:31.806268+0000","last_peered":"2024-09-19T07:31:31.806268+0000","last_clean":"2024-09-19T07:31:31.806268+0000","last_became_active":"2024-09-19T07:31:02.437034+0000","last_became_peered":"2024-09-19T07:31:02.437034+0000","last_unstale":"2024-09-19T07:31:31.806268+0000","last_undegraded":"2024-09-19T07:31:31.806268+0000","last_fullsized":"2024-09-19T07:31:31.806268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:55.310844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:51.895562+0000","last_change":"2024-09-19T07:30:59.410069+0000","last_active":"2024-09-19T07:31:51.895562+0000","last_peered":"2024-09-19T07:31:51.895562+0000","last_clean":"2024-09-19T07:31:51.895562+0000","last_became_active":"2024-09-19T07:30:59.409736+0000","last_became_peered":"2024-09-19T07:30:59.409736+0000","last_unstale":"2024-09-19T07:31:51.895562+0000","last_undegraded":"2024-09-19T07:31:51.895562+0000","last_fullsized":"2024-09-19T07:31:51.895562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:59:47.095227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":356,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531185+0000","last_change":"2024-09-19T07:29:38.994670+0000","last_active":"2024-09-19T07:31:32.531185+0000","last_peered":"2024-09-19T07:31:32.531185+0000","last_clean":"2024-09-19T07:31:32.531185+0000","last_became_active":"2024-09-19T07:29:38.994229+0000","last_became_peered":"2024-09-19T07:29:38.994229+0000","last_unstale":"2024-09-19T07:31:32.531185+0000","last_undegraded":"2024-09-19T07:31:32.531185+0000","last_fullsized":"2024-09-19T07:31:32.531185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:25:55.396577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:25:55.396577+0000","last_clean_scrub_stamp":"2024-09-19T07:25:55.396577+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:06.972254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531547+0000","last_change":"2024-09-19T07:31:12.497060+0000","last_active":"2024-09-19T07:31:32.531547+0000","last_peered":"2024-09-19T07:31:32.531547+0000","last_clean":"2024-09-19T07:31:32.531547+0000","last_became_active":"2024-09-19T07:31:12.496693+0000","last_became_peered":"2024-09-19T07:31:12.496693+0000","last_unstale":"2024-09-19T07:31:32.531547+0000","last_undegraded":"2024-09-19T07:31:32.531547+0000","last_fullsized":"2024-09-19T07:31:32.531547+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:41.879585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531603+0000","last_change":"2024-09-19T07:31:02.420732+0000","last_active":"2024-09-19T07:31:32.531603+0000","last_peered":"2024-09-19T07:31:32.531603+0000","last_clean":"2024-09-19T07:31:32.531603+0000","last_became_active":"2024-09-19T07:31:02.420394+0000","last_became_peered":"2024-09-19T07:31:02.420394+0000","last_unstale":"2024-09-19T07:31:32.531603+0000","last_undegraded":"2024-09-19T07:31:32.531603+0000","last_fullsized":"2024-09-19T07:31:32.531603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:48:36.157189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:50.896898+0000","last_change":"2024-09-19T07:31:00.027344+0000","last_active":"2024-09-19T07:31:50.896898+0000","last_peered":"2024-09-19T07:31:50.896898+0000","last_clean":"2024-09-19T07:31:50.896898+0000","last_became_active":"2024-09-19T07:31:00.026855+0000","last_became_peered":"2024-09-19T07:31:00.026855+0000","last_unstale":"2024-09-19T07:31:50.896898+0000","last_undegraded":"2024-09-19T07:31:50.896898+0000","last_fullsized":"2024-09-19T07:31:50.896898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:12.304161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.245452+0000","last_change":"2024-09-19T07:31:12.495883+0000","last_active":"2024-09-19T07:31:32.245452+0000","last_peered":"2024-09-19T07:31:32.245452+0000","last_clean":"2024-09-19T07:31:32.245452+0000","last_became_active":"2024-09-19T07:31:12.495253+0000","last_became_peered":"2024-09-19T07:31:12.495253+0000","last_unstale":"2024-09-19T07:31:32.245452+0000","last_undegraded":"2024-09-19T07:31:32.245452+0000","last_fullsized":"2024-09-19T07:31:32.245452+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:56.608977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531431+0000","last_change":"2024-09-19T07:31:02.427706+0000","last_active":"2024-09-19T07:31:32.531431+0000","last_peered":"2024-09-19T07:31:32.531431+0000","last_clean":"2024-09-19T07:31:32.531431+0000","last_became_active":"2024-09-19T07:31:02.427398+0000","last_became_peered":"2024-09-19T07:31:02.427398+0000","last_unstale":"2024-09-19T07:31:32.531431+0000","last_undegraded":"2024-09-19T07:31:32.531431+0000","last_fullsized":"2024-09-19T07:31:32.531431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:13.179472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.807839+0000","last_change":"2024-09-19T07:30:59.410020+0000","last_active":"2024-09-19T07:31:31.807839+0000","last_peered":"2024-09-19T07:31:31.807839+0000","last_clean":"2024-09-19T07:31:31.807839+0000","last_became_active":"2024-09-19T07:30:59.409633+0000","last_became_peered":"2024-09-19T07:30:59.409633+0000","last_unstale":"2024-09-19T07:31:31.807839+0000","last_undegraded":"2024-09-19T07:31:31.807839+0000","last_fullsized":"2024-09-19T07:31:31.807839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:49.673952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.019882+0000","last_change":"2024-09-19T07:31:12.511259+0000","last_active":"2024-09-19T07:31:32.019882+0000","last_peered":"2024-09-19T07:31:32.019882+0000","last_clean":"2024-09-19T07:31:32.019882+0000","last_became_active":"2024-09-19T07:31:12.510691+0000","last_became_peered":"2024-09-19T07:31:12.510691+0000","last_unstale":"2024-09-19T07:31:32.019882+0000","last_undegraded":"2024-09-19T07:31:32.019882+0000","last_fullsized":"2024-09-19T07:31:32.019882+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:09.388922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531697+0000","last_change":"2024-09-19T07:31:02.425922+0000","last_active":"2024-09-19T07:31:32.531697+0000","last_peered":"2024-09-19T07:31:32.531697+0000","last_clean":"2024-09-19T07:31:32.531697+0000","last_became_active":"2024-09-19T07:31:02.425565+0000","last_became_peered":"2024-09-19T07:31:02.425565+0000","last_unstale":"2024-09-19T07:31:32.531697+0000","last_undegraded":"2024-09-19T07:31:32.531697+0000","last_fullsized":"2024-09-19T07:31:32.531697+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:48.997362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":52,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.813529+0000","last_change":"2024-09-19T07:31:31.813529+0000","last_active":"2024-09-19T07:31:31.813529+0000","last_peered":"2024-09-19T07:31:31.813529+0000","last_clean":"2024-09-19T07:31:31.813529+0000","last_became_active":"2024-09-19T07:31:31.809205+0000","last_became_peered":"2024-09-19T07:31:31.809205+0000","last_unstale":"2024-09-19T07:31:31.813529+0000","last_undegraded":"2024-09-19T07:31:31.813529+0000","last_fullsized":"2024-09-19T07:31:31.813529+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:26.117905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806457+0000","last_change":"2024-09-19T07:31:12.497728+0000","last_active":"2024-09-19T07:31:31.806457+0000","last_peered":"2024-09-19T07:31:31.806457+0000","last_clean":"2024-09-19T07:31:31.806457+0000","last_became_active":"2024-09-19T07:31:12.497315+0000","last_became_peered":"2024-09-19T07:31:12.497315+0000","last_unstale":"2024-09-19T07:31:31.806457+0000","last_undegraded":"2024-09-19T07:31:31.806457+0000","last_fullsized":"2024-09-19T07:31:31.806457+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:12.174147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805094+0000","last_change":"2024-09-19T07:31:02.433205+0000","last_active":"2024-09-19T07:31:31.805094+0000","last_peered":"2024-09-19T07:31:31.805094+0000","last_clean":"2024-09-19T07:31:31.805094+0000","last_became_active":"2024-09-19T07:31:02.432856+0000","last_became_peered":"2024-09-19T07:31:02.432856+0000","last_unstale":"2024-09-19T07:31:31.805094+0000","last_undegraded":"2024-09-19T07:31:31.805094+0000","last_fullsized":"2024-09-19T07:31:31.805094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:31.470420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:51.890751+0000","last_change":"2024-09-19T07:30:59.411966+0000","last_active":"2024-09-19T07:31:51.890751+0000","last_peered":"2024-09-19T07:31:51.890751+0000","last_clean":"2024-09-19T07:31:51.890751+0000","last_became_active":"2024-09-19T07:30:59.411595+0000","last_became_peered":"2024-09-19T07:30:59.411595+0000","last_unstale":"2024-09-19T07:31:51.890751+0000","last_undegraded":"2024-09-19T07:31:51.890751+0000","last_fullsized":"2024-09-19T07:31:51.890751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:25.539059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806569+0000","last_change":"2024-09-19T07:31:12.500715+0000","last_active":"2024-09-19T07:31:31.806569+0000","last_peered":"2024-09-19T07:31:31.806569+0000","last_clean":"2024-09-19T07:31:31.806569+0000","last_became_active":"2024-09-19T07:31:12.500401+0000","last_became_peered":"2024-09-19T07:31:12.500401+0000","last_unstale":"2024-09-19T07:31:31.806569+0000","last_undegraded":"2024-09-19T07:31:31.806569+0000","last_fullsized":"2024-09-19T07:31:31.806569+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:59.158518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806740+0000","last_change":"2024-09-19T07:31:02.429062+0000","last_active":"2024-09-19T07:31:31.806740+0000","last_peered":"2024-09-19T07:31:31.806740+0000","last_clean":"2024-09-19T07:31:31.806740+0000","last_became_active":"2024-09-19T07:31:02.428703+0000","last_became_peered":"2024-09-19T07:31:02.428703+0000","last_unstale":"2024-09-19T07:31:31.806740+0000","last_undegraded":"2024-09-19T07:31:31.806740+0000","last_fullsized":"2024-09-19T07:31:31.806740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:30.928698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":55,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:50.897145+0000","last_change":"2024-09-19T07:30:59.444883+0000","last_active":"2024-09-19T07:31:50.897145+0000","last_peered":"2024-09-19T07:31:50.897145+0000","last_clean":"2024-09-19T07:31:50.897145+0000","last_became_active":"2024-09-19T07:30:59.444445+0000","last_became_peered":"2024-09-19T07:30:59.444445+0000","last_unstale":"2024-09-19T07:31:50.897145+0000","last_undegraded":"2024-09-19T07:31:50.897145+0000","last_fullsized":"2024-09-19T07:31:50.897145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:29:11.166450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803238+0000","last_change":"2024-09-19T07:31:12.508431+0000","last_active":"2024-09-19T07:31:31.803238+0000","last_peered":"2024-09-19T07:31:31.803238+0000","last_clean":"2024-09-19T07:31:31.803238+0000","last_became_active":"2024-09-19T07:31:12.507826+0000","last_became_peered":"2024-09-19T07:31:12.507826+0000","last_unstale":"2024-09-19T07:31:31.803238+0000","last_undegraded":"2024-09-19T07:31:31.803238+0000","last_fullsized":"2024-09-19T07:31:31.803238+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:59.545418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.530878+0000","last_change":"2024-09-19T07:31:02.423973+0000","last_active":"2024-09-19T07:31:32.530878+0000","last_peered":"2024-09-19T07:31:32.530878+0000","last_clean":"2024-09-19T07:31:32.530878+0000","last_became_active":"2024-09-19T07:31:02.423609+0000","last_became_peered":"2024-09-19T07:31:02.423609+0000","last_unstale":"2024-09-19T07:31:32.530878+0000","last_undegraded":"2024-09-19T07:31:32.530878+0000","last_fullsized":"2024-09-19T07:31:32.530878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:23:47.693286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"98'4","reported_seq":61,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:32:04.916610+0000","last_change":"2024-09-19T07:30:59.412862+0000","last_active":"2024-09-19T07:32:04.916610+0000","last_peered":"2024-09-19T07:32:04.916610+0000","last_clean":"2024-09-19T07:32:04.916610+0000","last_became_active":"2024-09-19T07:30:59.412401+0000","last_became_peered":"2024-09-19T07:30:59.412401+0000","last_unstale":"2024-09-19T07:32:04.916610+0000","last_undegraded":"2024-09-19T07:32:04.916610+0000","last_fullsized":"2024-09-19T07:32:04.916610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:49.461312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.808794+0000","last_change":"2024-09-19T07:31:12.507714+0000","last_active":"2024-09-19T07:31:31.808794+0000","last_peered":"2024-09-19T07:31:31.808794+0000","last_clean":"2024-09-19T07:31:31.808794+0000","last_became_active":"2024-09-19T07:31:12.506054+0000","last_became_peered":"2024-09-19T07:31:12.506054+0000","last_unstale":"2024-09-19T07:31:31.808794+0000","last_undegraded":"2024-09-19T07:31:31.808794+0000","last_fullsized":"2024-09-19T07:31:31.808794+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:18.070858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531335+0000","last_change":"2024-09-19T07:31:02.424718+0000","last_active":"2024-09-19T07:31:32.531335+0000","last_peered":"2024-09-19T07:31:32.531335+0000","last_clean":"2024-09-19T07:31:32.531335+0000","last_became_active":"2024-09-19T07:31:02.424351+0000","last_became_peered":"2024-09-19T07:31:02.424351+0000","last_unstale":"2024-09-19T07:31:32.531335+0000","last_undegraded":"2024-09-19T07:31:32.531335+0000","last_fullsized":"2024-09-19T07:31:32.531335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:59.396752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:49.886069+0000","last_change":"2024-09-19T07:30:59.410551+0000","last_active":"2024-09-19T07:31:49.886069+0000","last_peered":"2024-09-19T07:31:49.886069+0000","last_clean":"2024-09-19T07:31:49.886069+0000","last_became_active":"2024-09-19T07:30:59.410226+0000","last_became_peered":"2024-09-19T07:30:59.410226+0000","last_unstale":"2024-09-19T07:31:49.886069+0000","last_undegraded":"2024-09-19T07:31:49.886069+0000","last_fullsized":"2024-09-19T07:31:49.886069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:18.815424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.532426+0000","last_change":"2024-09-19T07:31:32.532426+0000","last_active":"2024-09-19T07:31:32.532426+0000","last_peered":"2024-09-19T07:31:32.532426+0000","last_clean":"2024-09-19T07:31:32.532426+0000","last_became_active":"2024-09-19T07:31:32.532034+0000","last_became_peered":"2024-09-19T07:31:32.532034+0000","last_unstale":"2024-09-19T07:31:32.532426+0000","last_undegraded":"2024-09-19T07:31:32.532426+0000","last_fullsized":"2024-09-19T07:31:32.532426+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:09.421369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.802848+0000","last_change":"2024-09-19T07:31:02.444240+0000","last_active":"2024-09-19T07:31:31.802848+0000","last_peered":"2024-09-19T07:31:31.802848+0000","last_clean":"2024-09-19T07:31:31.802848+0000","last_became_active":"2024-09-19T07:31:02.443808+0000","last_became_peered":"2024-09-19T07:31:02.443808+0000","last_unstale":"2024-09-19T07:31:31.802848+0000","last_undegraded":"2024-09-19T07:31:31.802848+0000","last_fullsized":"2024-09-19T07:31:31.802848+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:12.898708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:50.896980+0000","last_change":"2024-09-19T07:30:59.411130+0000","last_active":"2024-09-19T07:31:50.896980+0000","last_peered":"2024-09-19T07:31:50.896980+0000","last_clean":"2024-09-19T07:31:50.896980+0000","last_became_active":"2024-09-19T07:30:59.410837+0000","last_became_peered":"2024-09-19T07:30:59.410837+0000","last_unstale":"2024-09-19T07:31:50.896980+0000","last_undegraded":"2024-09-19T07:31:50.896980+0000","last_fullsized":"2024-09-19T07:31:50.896980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:04.194868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.248768+0000","last_change":"2024-09-19T07:31:32.248185+0000","last_active":"2024-09-19T07:31:32.248768+0000","last_peered":"2024-09-19T07:31:32.248768+0000","last_clean":"2024-09-19T07:31:32.248768+0000","last_became_active":"2024-09-19T07:31:32.247514+0000","last_became_peered":"2024-09-19T07:31:32.247514+0000","last_unstale":"2024-09-19T07:31:32.248768+0000","last_undegraded":"2024-09-19T07:31:32.248768+0000","last_fullsized":"2024-09-19T07:31:32.248768+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:09.713374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.246695+0000","last_change":"2024-09-19T07:31:02.421038+0000","last_active":"2024-09-19T07:31:32.246695+0000","last_peered":"2024-09-19T07:31:32.246695+0000","last_clean":"2024-09-19T07:31:32.246695+0000","last_became_active":"2024-09-19T07:31:02.420715+0000","last_became_peered":"2024-09-19T07:31:02.420715+0000","last_unstale":"2024-09-19T07:31:32.246695+0000","last_undegraded":"2024-09-19T07:31:32.246695+0000","last_fullsized":"2024-09-19T07:31:32.246695+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:39.943123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"98'26","reported_seq":98,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:32:04.916793+0000","last_change":"2024-09-19T07:30:59.414095+0000","last_active":"2024-09-19T07:32:04.916793+0000","last_peered":"2024-09-19T07:32:04.916793+0000","last_clean":"2024-09-19T07:32:04.916793+0000","last_became_active":"2024-09-19T07:30:59.413431+0000","last_became_peered":"2024-09-19T07:30:59.413431+0000","last_unstale":"2024-09-19T07:32:04.916793+0000","last_undegraded":"2024-09-19T07:32:04.916793+0000","last_fullsized":"2024-09-19T07:32:04.916793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:00.573369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49098,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":64,"num_write":27,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.808429+0000","last_change":"2024-09-19T07:31:12.500563+0000","last_active":"2024-09-19T07:31:31.808429+0000","last_peered":"2024-09-19T07:31:31.808429+0000","last_clean":"2024-09-19T07:31:31.808429+0000","last_became_active":"2024-09-19T07:31:12.500170+0000","last_became_peered":"2024-09-19T07:31:12.500170+0000","last_unstale":"2024-09-19T07:31:31.808429+0000","last_undegraded":"2024-09-19T07:31:31.808429+0000","last_fullsized":"2024-09-19T07:31:31.808429+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:17.897055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.245660+0000","last_change":"2024-09-19T07:31:02.431931+0000","last_active":"2024-09-19T07:31:32.245660+0000","last_peered":"2024-09-19T07:31:32.245660+0000","last_clean":"2024-09-19T07:31:32.245660+0000","last_became_active":"2024-09-19T07:31:02.431579+0000","last_became_peered":"2024-09-19T07:31:02.431579+0000","last_unstale":"2024-09-19T07:31:32.245660+0000","last_undegraded":"2024-09-19T07:31:32.245660+0000","last_fullsized":"2024-09-19T07:31:32.245660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:18.260616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:49.885400+0000","last_change":"2024-09-19T07:30:59.445150+0000","last_active":"2024-09-19T07:31:49.885400+0000","last_peered":"2024-09-19T07:31:49.885400+0000","last_clean":"2024-09-19T07:31:49.885400+0000","last_became_active":"2024-09-19T07:30:59.444776+0000","last_became_peered":"2024-09-19T07:30:59.444776+0000","last_unstale":"2024-09-19T07:31:49.885400+0000","last_undegraded":"2024-09-19T07:31:49.885400+0000","last_fullsized":"2024-09-19T07:31:49.885400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:30.654652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.248246+0000","last_change":"2024-09-19T07:31:32.248246+0000","last_active":"2024-09-19T07:31:32.248246+0000","last_peered":"2024-09-19T07:31:32.248246+0000","last_clean":"2024-09-19T07:31:32.248246+0000","last_became_active":"2024-09-19T07:31:32.247456+0000","last_became_peered":"2024-09-19T07:31:32.247456+0000","last_unstale":"2024-09-19T07:31:32.248246+0000","last_undegraded":"2024-09-19T07:31:32.248246+0000","last_fullsized":"2024-09-19T07:31:32.248246+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:31:44.320026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.020209+0000","last_change":"2024-09-19T07:31:02.441924+0000","last_active":"2024-09-19T07:31:32.020209+0000","last_peered":"2024-09-19T07:31:32.020209+0000","last_clean":"2024-09-19T07:31:32.020209+0000","last_became_active":"2024-09-19T07:31:02.438247+0000","last_became_peered":"2024-09-19T07:31:02.438247+0000","last_unstale":"2024-09-19T07:31:32.020209+0000","last_undegraded":"2024-09-19T07:31:32.020209+0000","last_fullsized":"2024-09-19T07:31:32.020209+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:12.878185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:49.884781+0000","last_change":"2024-09-19T07:30:59.409679+0000","last_active":"2024-09-19T07:31:49.884781+0000","last_peered":"2024-09-19T07:31:49.884781+0000","last_clean":"2024-09-19T07:31:49.884781+0000","last_became_active":"2024-09-19T07:30:59.409313+0000","last_became_peered":"2024-09-19T07:30:59.409313+0000","last_unstale":"2024-09-19T07:31:49.884781+0000","last_undegraded":"2024-09-19T07:31:49.884781+0000","last_fullsized":"2024-09-19T07:31:49.884781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:42.121358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806947+0000","last_change":"2024-09-19T07:31:12.506596+0000","last_active":"2024-09-19T07:31:31.806947+0000","last_peered":"2024-09-19T07:31:31.806947+0000","last_clean":"2024-09-19T07:31:31.806947+0000","last_became_active":"2024-09-19T07:31:12.505107+0000","last_became_peered":"2024-09-19T07:31:12.505107+0000","last_unstale":"2024-09-19T07:31:31.806947+0000","last_undegraded":"2024-09-19T07:31:31.806947+0000","last_fullsized":"2024-09-19T07:31:31.806947+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:39.157988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.246037+0000","last_change":"2024-09-19T07:31:02.429883+0000","last_active":"2024-09-19T07:31:32.246037+0000","last_peered":"2024-09-19T07:31:32.246037+0000","last_clean":"2024-09-19T07:31:32.246037+0000","last_became_active":"2024-09-19T07:31:02.429554+0000","last_became_peered":"2024-09-19T07:31:02.429554+0000","last_unstale":"2024-09-19T07:31:32.246037+0000","last_undegraded":"2024-09-19T07:31:32.246037+0000","last_fullsized":"2024-09-19T07:31:32.246037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:00.336506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:32:03.914786+0000","last_change":"2024-09-19T07:30:59.414735+0000","last_active":"2024-09-19T07:32:03.914786+0000","last_peered":"2024-09-19T07:32:03.914786+0000","last_clean":"2024-09-19T07:32:03.914786+0000","last_became_active":"2024-09-19T07:30:59.414385+0000","last_became_peered":"2024-09-19T07:30:59.414385+0000","last_unstale":"2024-09-19T07:32:03.914786+0000","last_undegraded":"2024-09-19T07:32:03.914786+0000","last_fullsized":"2024-09-19T07:32:03.914786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:43.668794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805429+0000","last_change":"2024-09-19T07:31:12.497242+0000","last_active":"2024-09-19T07:31:31.805429+0000","last_peered":"2024-09-19T07:31:31.805429+0000","last_clean":"2024-09-19T07:31:31.805429+0000","last_became_active":"2024-09-19T07:31:12.496857+0000","last_became_peered":"2024-09-19T07:31:12.496857+0000","last_unstale":"2024-09-19T07:31:31.805429+0000","last_undegraded":"2024-09-19T07:31:31.805429+0000","last_fullsized":"2024-09-19T07:31:31.805429+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:18.418594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.020041+0000","last_change":"2024-09-19T07:31:12.504507+0000","last_active":"2024-09-19T07:31:32.020041+0000","last_peered":"2024-09-19T07:31:32.020041+0000","last_clean":"2024-09-19T07:31:32.020041+0000","last_became_active":"2024-09-19T07:31:12.503289+0000","last_became_peered":"2024-09-19T07:31:12.503289+0000","last_unstale":"2024-09-19T07:31:32.020041+0000","last_undegraded":"2024-09-19T07:31:32.020041+0000","last_fullsized":"2024-09-19T07:31:32.020041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:21.771692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.807713+0000","last_change":"2024-09-19T07:31:02.429634+0000","last_active":"2024-09-19T07:31:31.807713+0000","last_peered":"2024-09-19T07:31:31.807713+0000","last_clean":"2024-09-19T07:31:31.807713+0000","last_became_active":"2024-09-19T07:31:02.429280+0000","last_became_peered":"2024-09-19T07:31:02.429280+0000","last_unstale":"2024-09-19T07:31:31.807713+0000","last_undegraded":"2024-09-19T07:31:31.807713+0000","last_fullsized":"2024-09-19T07:31:31.807713+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:13.872594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.807160+0000","last_change":"2024-09-19T07:30:59.409961+0000","last_active":"2024-09-19T07:31:31.807160+0000","last_peered":"2024-09-19T07:31:31.807160+0000","last_clean":"2024-09-19T07:31:31.807160+0000","last_became_active":"2024-09-19T07:30:59.409582+0000","last_became_peered":"2024-09-19T07:30:59.409582+0000","last_unstale":"2024-09-19T07:31:31.807160+0000","last_undegraded":"2024-09-19T07:31:31.807160+0000","last_fullsized":"2024-09-19T07:31:31.807160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:31.075378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.804966+0000","last_change":"2024-09-19T07:31:12.510999+0000","last_active":"2024-09-19T07:31:31.804966+0000","last_peered":"2024-09-19T07:31:31.804966+0000","last_clean":"2024-09-19T07:31:31.804966+0000","last_became_active":"2024-09-19T07:31:12.510604+0000","last_became_peered":"2024-09-19T07:31:12.510604+0000","last_unstale":"2024-09-19T07:31:31.804966+0000","last_undegraded":"2024-09-19T07:31:31.804966+0000","last_fullsized":"2024-09-19T07:31:31.804966+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:04.779289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805444+0000","last_change":"2024-09-19T07:31:02.428108+0000","last_active":"2024-09-19T07:31:31.805444+0000","last_peered":"2024-09-19T07:31:31.805444+0000","last_clean":"2024-09-19T07:31:31.805444+0000","last_became_active":"2024-09-19T07:31:02.425537+0000","last_became_peered":"2024-09-19T07:31:02.425537+0000","last_unstale":"2024-09-19T07:31:31.805444+0000","last_undegraded":"2024-09-19T07:31:31.805444+0000","last_fullsized":"2024-09-19T07:31:31.805444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:36.094089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:50.896840+0000","last_change":"2024-09-19T07:30:59.404687+0000","last_active":"2024-09-19T07:31:50.896840+0000","last_peered":"2024-09-19T07:31:50.896840+0000","last_clean":"2024-09-19T07:31:50.896840+0000","last_became_active":"2024-09-19T07:30:59.404222+0000","last_became_peered":"2024-09-19T07:30:59.404222+0000","last_unstale":"2024-09-19T07:31:50.896840+0000","last_undegraded":"2024-09-19T07:31:50.896840+0000","last_fullsized":"2024-09-19T07:31:50.896840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:11.287464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:50.895495+0000","last_change":"2024-09-19T07:30:59.414711+0000","last_active":"2024-09-19T07:31:50.895495+0000","last_peered":"2024-09-19T07:31:50.895495+0000","last_clean":"2024-09-19T07:31:50.895495+0000","last_became_active":"2024-09-19T07:30:59.414349+0000","last_became_peered":"2024-09-19T07:30:59.414349+0000","last_unstale":"2024-09-19T07:31:50.895495+0000","last_undegraded":"2024-09-19T07:31:50.895495+0000","last_fullsized":"2024-09-19T07:31:50.895495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:57:09.910328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.810970+0000","last_change":"2024-09-19T07:31:31.810186+0000","last_active":"2024-09-19T07:31:31.810970+0000","last_peered":"2024-09-19T07:31:31.810970+0000","last_clean":"2024-09-19T07:31:31.810970+0000","last_became_active":"2024-09-19T07:31:31.809861+0000","last_became_peered":"2024-09-19T07:31:31.809861+0000","last_unstale":"2024-09-19T07:31:31.810970+0000","last_undegraded":"2024-09-19T07:31:31.810970+0000","last_fullsized":"2024-09-19T07:31:31.810970+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:49.443012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803860+0000","last_change":"2024-09-19T07:31:12.506060+0000","last_active":"2024-09-19T07:31:31.803860+0000","last_peered":"2024-09-19T07:31:31.803860+0000","last_clean":"2024-09-19T07:31:31.803860+0000","last_became_active":"2024-09-19T07:31:12.505535+0000","last_became_peered":"2024-09-19T07:31:12.505535+0000","last_unstale":"2024-09-19T07:31:31.803860+0000","last_undegraded":"2024-09-19T07:31:31.803860+0000","last_fullsized":"2024-09-19T07:31:31.803860+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:02.820475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'6","reported_seq":62,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:32:03.915310+0000","last_change":"2024-09-19T07:30:59.408594+0000","last_active":"2024-09-19T07:32:03.915310+0000","last_peered":"2024-09-19T07:32:03.915310+0000","last_clean":"2024-09-19T07:32:03.915310+0000","last_became_active":"2024-09-19T07:30:59.408164+0000","last_became_peered":"2024-09-19T07:30:59.408164+0000","last_unstale":"2024-09-19T07:32:03.915310+0000","last_undegraded":"2024-09-19T07:32:03.915310+0000","last_fullsized":"2024-09-19T07:32:03.915310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:23.950637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.246515+0000","last_change":"2024-09-19T07:31:02.419940+0000","last_active":"2024-09-19T07:31:32.246515+0000","last_peered":"2024-09-19T07:31:32.246515+0000","last_clean":"2024-09-19T07:31:32.246515+0000","last_became_active":"2024-09-19T07:31:02.419618+0000","last_became_peered":"2024-09-19T07:31:02.419618+0000","last_unstale":"2024-09-19T07:31:32.246515+0000","last_undegraded":"2024-09-19T07:31:32.246515+0000","last_fullsized":"2024-09-19T07:31:32.246515+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:18.208200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803990+0000","last_change":"2024-09-19T07:31:12.497271+0000","last_active":"2024-09-19T07:31:31.803990+0000","last_peered":"2024-09-19T07:31:31.803990+0000","last_clean":"2024-09-19T07:31:31.803990+0000","last_became_active":"2024-09-19T07:31:12.496882+0000","last_became_peered":"2024-09-19T07:31:12.496882+0000","last_unstale":"2024-09-19T07:31:31.803990+0000","last_undegraded":"2024-09-19T07:31:31.803990+0000","last_fullsized":"2024-09-19T07:31:31.803990+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:48.714941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.804672+0000","last_change":"2024-09-19T07:31:02.429917+0000","last_active":"2024-09-19T07:31:31.804672+0000","last_peered":"2024-09-19T07:31:31.804672+0000","last_clean":"2024-09-19T07:31:31.804672+0000","last_became_active":"2024-09-19T07:31:02.429535+0000","last_became_peered":"2024-09-19T07:31:02.429535+0000","last_unstale":"2024-09-19T07:31:31.804672+0000","last_undegraded":"2024-09-19T07:31:31.804672+0000","last_fullsized":"2024-09-19T07:31:31.804672+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:21.958491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803870+0000","last_change":"2024-09-19T07:31:02.433135+0000","last_active":"2024-09-19T07:31:31.803870+0000","last_peered":"2024-09-19T07:31:31.803870+0000","last_clean":"2024-09-19T07:31:31.803870+0000","last_became_active":"2024-09-19T07:31:02.432800+0000","last_became_peered":"2024-09-19T07:31:02.432800+0000","last_unstale":"2024-09-19T07:31:31.803870+0000","last_undegraded":"2024-09-19T07:31:31.803870+0000","last_fullsized":"2024-09-19T07:31:31.803870+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:01.559565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803436+0000","last_change":"2024-09-19T07:31:02.438670+0000","last_active":"2024-09-19T07:31:31.803436+0000","last_peered":"2024-09-19T07:31:31.803436+0000","last_clean":"2024-09-19T07:31:31.803436+0000","last_became_active":"2024-09-19T07:31:02.438191+0000","last_became_peered":"2024-09-19T07:31:02.438191+0000","last_unstale":"2024-09-19T07:31:31.803436+0000","last_undegraded":"2024-09-19T07:31:31.803436+0000","last_fullsized":"2024-09-19T07:31:31.803436+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:55.317996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.019815+0000","last_change":"2024-09-19T07:31:02.434826+0000","last_active":"2024-09-19T07:31:32.019815+0000","last_peered":"2024-09-19T07:31:32.019815+0000","last_clean":"2024-09-19T07:31:32.019815+0000","last_became_active":"2024-09-19T07:31:02.433695+0000","last_became_peered":"2024-09-19T07:31:02.433695+0000","last_unstale":"2024-09-19T07:31:32.019815+0000","last_undegraded":"2024-09-19T07:31:32.019815+0000","last_fullsized":"2024-09-19T07:31:32.019815+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:33:28.866345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.021091+0000","last_change":"2024-09-19T07:31:32.021091+0000","last_active":"2024-09-19T07:31:32.021091+0000","last_peered":"2024-09-19T07:31:32.021091+0000","last_clean":"2024-09-19T07:31:32.021091+0000","last_became_active":"2024-09-19T07:31:32.020537+0000","last_became_peered":"2024-09-19T07:31:32.020537+0000","last_unstale":"2024-09-19T07:31:32.021091+0000","last_undegraded":"2024-09-19T07:31:32.021091+0000","last_fullsized":"2024-09-19T07:31:32.021091+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:31.649790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805242+0000","last_change":"2024-09-19T07:31:02.424514+0000","last_active":"2024-09-19T07:31:31.805242+0000","last_peered":"2024-09-19T07:31:31.805242+0000","last_clean":"2024-09-19T07:31:31.805242+0000","last_became_active":"2024-09-19T07:31:02.424194+0000","last_became_peered":"2024-09-19T07:31:02.424194+0000","last_unstale":"2024-09-19T07:31:31.805242+0000","last_undegraded":"2024-09-19T07:31:31.805242+0000","last_fullsized":"2024-09-19T07:31:31.805242+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:41:47.039372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803999+0000","last_change":"2024-09-19T07:31:02.429624+0000","last_active":"2024-09-19T07:31:31.803999+0000","last_peered":"2024-09-19T07:31:31.803999+0000","last_clean":"2024-09-19T07:31:31.803999+0000","last_became_active":"2024-09-19T07:31:02.429316+0000","last_became_peered":"2024-09-19T07:31:02.429316+0000","last_unstale":"2024-09-19T07:31:31.803999+0000","last_undegraded":"2024-09-19T07:31:31.803999+0000","last_fullsized":"2024-09-19T07:31:31.803999+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:15.736433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.811843+0000","last_change":"2024-09-19T07:31:31.810511+0000","last_active":"2024-09-19T07:31:31.811843+0000","last_peered":"2024-09-19T07:31:31.811843+0000","last_clean":"2024-09-19T07:31:31.811843+0000","last_became_active":"2024-09-19T07:31:31.809805+0000","last_became_peered":"2024-09-19T07:31:31.809805+0000","last_unstale":"2024-09-19T07:31:31.811843+0000","last_undegraded":"2024-09-19T07:31:31.811843+0000","last_fullsized":"2024-09-19T07:31:31.811843+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:48.457693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806855+0000","last_change":"2024-09-19T07:31:02.433859+0000","last_active":"2024-09-19T07:31:31.806855+0000","last_peered":"2024-09-19T07:31:31.806855+0000","last_clean":"2024-09-19T07:31:31.806855+0000","last_became_active":"2024-09-19T07:31:02.433547+0000","last_became_peered":"2024-09-19T07:31:02.433547+0000","last_unstale":"2024-09-19T07:31:31.806855+0000","last_undegraded":"2024-09-19T07:31:31.806855+0000","last_fullsized":"2024-09-19T07:31:31.806855+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:09.623326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805944+0000","last_change":"2024-09-19T07:31:02.442174+0000","last_active":"2024-09-19T07:31:31.805944+0000","last_peered":"2024-09-19T07:31:31.805944+0000","last_clean":"2024-09-19T07:31:31.805944+0000","last_became_active":"2024-09-19T07:31:02.441851+0000","last_became_peered":"2024-09-19T07:31:02.441851+0000","last_unstale":"2024-09-19T07:31:31.805944+0000","last_undegraded":"2024-09-19T07:31:31.805944+0000","last_fullsized":"2024-09-19T07:31:31.805944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:59.235498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.245700+0000","last_change":"2024-09-19T07:31:02.439526+0000","last_active":"2024-09-19T07:31:32.245700+0000","last_peered":"2024-09-19T07:31:32.245700+0000","last_clean":"2024-09-19T07:31:32.245700+0000","last_became_active":"2024-09-19T07:31:02.438996+0000","last_became_peered":"2024-09-19T07:31:02.438996+0000","last_unstale":"2024-09-19T07:31:32.245700+0000","last_undegraded":"2024-09-19T07:31:32.245700+0000","last_fullsized":"2024-09-19T07:31:32.245700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:55.909727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803030+0000","last_change":"2024-09-19T07:31:02.427770+0000","last_active":"2024-09-19T07:31:31.803030+0000","last_peered":"2024-09-19T07:31:31.803030+0000","last_clean":"2024-09-19T07:31:31.803030+0000","last_became_active":"2024-09-19T07:31:02.427208+0000","last_became_peered":"2024-09-19T07:31:02.427208+0000","last_unstale":"2024-09-19T07:31:31.803030+0000","last_undegraded":"2024-09-19T07:31:31.803030+0000","last_fullsized":"2024-09-19T07:31:31.803030+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:01.171781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805785+0000","last_change":"2024-09-19T07:31:02.433089+0000","last_active":"2024-09-19T07:31:31.805785+0000","last_peered":"2024-09-19T07:31:31.805785+0000","last_clean":"2024-09-19T07:31:31.805785+0000","last_became_active":"2024-09-19T07:31:02.432744+0000","last_became_peered":"2024-09-19T07:31:02.432744+0000","last_unstale":"2024-09-19T07:31:31.805785+0000","last_undegraded":"2024-09-19T07:31:31.805785+0000","last_fullsized":"2024-09-19T07:31:31.805785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:32.959555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.804158+0000","last_change":"2024-09-19T07:31:02.435482+0000","last_active":"2024-09-19T07:31:31.804158+0000","last_peered":"2024-09-19T07:31:31.804158+0000","last_clean":"2024-09-19T07:31:31.804158+0000","last_became_active":"2024-09-19T07:31:02.435164+0000","last_became_peered":"2024-09-19T07:31:02.435164+0000","last_unstale":"2024-09-19T07:31:31.804158+0000","last_undegraded":"2024-09-19T07:31:31.804158+0000","last_fullsized":"2024-09-19T07:31:31.804158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:54.478049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.809318+0000","last_change":"2024-09-19T07:31:02.426768+0000","last_active":"2024-09-19T07:31:31.809318+0000","last_peered":"2024-09-19T07:31:31.809318+0000","last_clean":"2024-09-19T07:31:31.809318+0000","last_became_active":"2024-09-19T07:31:02.425592+0000","last_became_peered":"2024-09-19T07:31:02.425592+0000","last_unstale":"2024-09-19T07:31:31.809318+0000","last_undegraded":"2024-09-19T07:31:31.809318+0000","last_fullsized":"2024-09-19T07:31:31.809318+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:44.985532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806121+0000","last_change":"2024-09-19T07:31:02.426233+0000","last_active":"2024-09-19T07:31:31.806121+0000","last_peered":"2024-09-19T07:31:31.806121+0000","last_clean":"2024-09-19T07:31:31.806121+0000","last_became_active":"2024-09-19T07:31:02.425777+0000","last_became_peered":"2024-09-19T07:31:02.425777+0000","last_unstale":"2024-09-19T07:31:31.806121+0000","last_undegraded":"2024-09-19T07:31:31.806121+0000","last_fullsized":"2024-09-19T07:31:31.806121+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:31.497641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.245261+0000","last_change":"2024-09-19T07:31:02.428671+0000","last_active":"2024-09-19T07:31:32.245261+0000","last_peered":"2024-09-19T07:31:32.245261+0000","last_clean":"2024-09-19T07:31:32.245261+0000","last_became_active":"2024-09-19T07:31:02.425930+0000","last_became_peered":"2024-09-19T07:31:02.425930+0000","last_unstale":"2024-09-19T07:31:32.245261+0000","last_undegraded":"2024-09-19T07:31:32.245261+0000","last_fullsized":"2024-09-19T07:31:32.245261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:21.721694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.019802+0000","last_change":"2024-09-19T07:31:02.441450+0000","last_active":"2024-09-19T07:31:32.019802+0000","last_peered":"2024-09-19T07:31:32.019802+0000","last_clean":"2024-09-19T07:31:32.019802+0000","last_became_active":"2024-09-19T07:31:02.436673+0000","last_became_peered":"2024-09-19T07:31:02.436673+0000","last_unstale":"2024-09-19T07:31:32.019802+0000","last_undegraded":"2024-09-19T07:31:32.019802+0000","last_fullsized":"2024-09-19T07:31:32.019802+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:29.744665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.019522+0000","last_change":"2024-09-19T07:31:02.427449+0000","last_active":"2024-09-19T07:31:32.019522+0000","last_peered":"2024-09-19T07:31:32.019522+0000","last_clean":"2024-09-19T07:31:32.019522+0000","last_became_active":"2024-09-19T07:31:02.426601+0000","last_became_peered":"2024-09-19T07:31:02.426601+0000","last_unstale":"2024-09-19T07:31:32.019522+0000","last_undegraded":"2024-09-19T07:31:32.019522+0000","last_fullsized":"2024-09-19T07:31:32.019522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:36.752529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803008+0000","last_change":"2024-09-19T07:31:02.424816+0000","last_active":"2024-09-19T07:31:31.803008+0000","last_peered":"2024-09-19T07:31:31.803008+0000","last_clean":"2024-09-19T07:31:31.803008+0000","last_became_active":"2024-09-19T07:31:02.423729+0000","last_became_peered":"2024-09-19T07:31:02.423729+0000","last_unstale":"2024-09-19T07:31:31.803008+0000","last_undegraded":"2024-09-19T07:31:31.803008+0000","last_fullsized":"2024-09-19T07:31:31.803008+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:39.546052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806843+0000","last_change":"2024-09-19T07:31:02.428444+0000","last_active":"2024-09-19T07:31:31.806843+0000","last_peered":"2024-09-19T07:31:31.806843+0000","last_clean":"2024-09-19T07:31:31.806843+0000","last_became_active":"2024-09-19T07:31:02.428121+0000","last_became_peered":"2024-09-19T07:31:02.428121+0000","last_unstale":"2024-09-19T07:31:31.806843+0000","last_undegraded":"2024-09-19T07:31:31.806843+0000","last_fullsized":"2024-09-19T07:31:31.806843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:52.649412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531604+0000","last_change":"2024-09-19T07:31:02.423102+0000","last_active":"2024-09-19T07:31:32.531604+0000","last_peered":"2024-09-19T07:31:32.531604+0000","last_clean":"2024-09-19T07:31:32.531604+0000","last_became_active":"2024-09-19T07:31:02.422672+0000","last_became_peered":"2024-09-19T07:31:02.422672+0000","last_unstale":"2024-09-19T07:31:32.531604+0000","last_undegraded":"2024-09-19T07:31:32.531604+0000","last_fullsized":"2024-09-19T07:31:32.531604+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:45.733320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806031+0000","last_change":"2024-09-19T07:31:02.434599+0000","last_active":"2024-09-19T07:31:31.806031+0000","last_peered":"2024-09-19T07:31:31.806031+0000","last_clean":"2024-09-19T07:31:31.806031+0000","last_became_active":"2024-09-19T07:31:02.434263+0000","last_became_peered":"2024-09-19T07:31:02.434263+0000","last_unstale":"2024-09-19T07:31:31.806031+0000","last_undegraded":"2024-09-19T07:31:31.806031+0000","last_fullsized":"2024-09-19T07:31:31.806031+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:16.104483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.246099+0000","last_change":"2024-09-19T07:31:02.429278+0000","last_active":"2024-09-19T07:31:32.246099+0000","last_peered":"2024-09-19T07:31:32.246099+0000","last_clean":"2024-09-19T07:31:32.246099+0000","last_became_active":"2024-09-19T07:31:02.428991+0000","last_became_peered":"2024-09-19T07:31:02.428991+0000","last_unstale":"2024-09-19T07:31:32.246099+0000","last_undegraded":"2024-09-19T07:31:32.246099+0000","last_fullsized":"2024-09-19T07:31:32.246099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:02.421684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805392+0000","last_change":"2024-09-19T07:31:02.436412+0000","last_active":"2024-09-19T07:31:31.805392+0000","last_peered":"2024-09-19T07:31:31.805392+0000","last_clean":"2024-09-19T07:31:31.805392+0000","last_became_active":"2024-09-19T07:31:02.436101+0000","last_became_peered":"2024-09-19T07:31:02.436101+0000","last_unstale":"2024-09-19T07:31:31.805392+0000","last_undegraded":"2024-09-19T07:31:31.805392+0000","last_fullsized":"2024-09-19T07:31:31.805392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:34.827734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531039+0000","last_change":"2024-09-19T07:31:02.423414+0000","last_active":"2024-09-19T07:31:32.531039+0000","last_peered":"2024-09-19T07:31:32.531039+0000","last_clean":"2024-09-19T07:31:32.531039+0000","last_became_active":"2024-09-19T07:31:02.423100+0000","last_became_peered":"2024-09-19T07:31:02.423100+0000","last_unstale":"2024-09-19T07:31:32.531039+0000","last_undegraded":"2024-09-19T07:31:32.531039+0000","last_fullsized":"2024-09-19T07:31:32.531039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:15.269095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805431+0000","last_change":"2024-09-19T07:31:02.434953+0000","last_active":"2024-09-19T07:31:31.805431+0000","last_peered":"2024-09-19T07:31:31.805431+0000","last_clean":"2024-09-19T07:31:31.805431+0000","last_became_active":"2024-09-19T07:31:02.434662+0000","last_became_peered":"2024-09-19T07:31:02.434662+0000","last_unstale":"2024-09-19T07:31:31.805431+0000","last_undegraded":"2024-09-19T07:31:31.805431+0000","last_fullsized":"2024-09-19T07:31:31.805431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:17.132108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.246334+0000","last_change":"2024-09-19T07:31:02.427959+0000","last_active":"2024-09-19T07:31:32.246334+0000","last_peered":"2024-09-19T07:31:32.246334+0000","last_clean":"2024-09-19T07:31:32.246334+0000","last_became_active":"2024-09-19T07:31:02.427671+0000","last_became_peered":"2024-09-19T07:31:02.427671+0000","last_unstale":"2024-09-19T07:31:32.246334+0000","last_undegraded":"2024-09-19T07:31:32.246334+0000","last_fullsized":"2024-09-19T07:31:32.246334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:19.608005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806804+0000","last_change":"2024-09-19T07:31:02.433310+0000","last_active":"2024-09-19T07:31:31.806804+0000","last_peered":"2024-09-19T07:31:31.806804+0000","last_clean":"2024-09-19T07:31:31.806804+0000","last_became_active":"2024-09-19T07:31:02.432922+0000","last_became_peered":"2024-09-19T07:31:02.432922+0000","last_unstale":"2024-09-19T07:31:31.806804+0000","last_undegraded":"2024-09-19T07:31:31.806804+0000","last_fullsized":"2024-09-19T07:31:31.806804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:30.043075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.019884+0000","last_change":"2024-09-19T07:31:02.429452+0000","last_active":"2024-09-19T07:31:32.019884+0000","last_peered":"2024-09-19T07:31:32.019884+0000","last_clean":"2024-09-19T07:31:32.019884+0000","last_became_active":"2024-09-19T07:31:02.429142+0000","last_became_peered":"2024-09-19T07:31:02.429142+0000","last_unstale":"2024-09-19T07:31:32.019884+0000","last_undegraded":"2024-09-19T07:31:32.019884+0000","last_fullsized":"2024-09-19T07:31:32.019884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:42.107179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.020039+0000","last_change":"2024-09-19T07:31:02.431197+0000","last_active":"2024-09-19T07:31:32.020039+0000","last_peered":"2024-09-19T07:31:32.020039+0000","last_clean":"2024-09-19T07:31:32.020039+0000","last_became_active":"2024-09-19T07:31:02.430812+0000","last_became_peered":"2024-09-19T07:31:02.430812+0000","last_unstale":"2024-09-19T07:31:32.020039+0000","last_undegraded":"2024-09-19T07:31:32.020039+0000","last_fullsized":"2024-09-19T07:31:32.020039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:10.682008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806102+0000","last_change":"2024-09-19T07:31:02.426753+0000","last_active":"2024-09-19T07:31:31.806102+0000","last_peered":"2024-09-19T07:31:31.806102+0000","last_clean":"2024-09-19T07:31:31.806102+0000","last_became_active":"2024-09-19T07:31:02.425870+0000","last_became_peered":"2024-09-19T07:31:02.425870+0000","last_unstale":"2024-09-19T07:31:31.806102+0000","last_undegraded":"2024-09-19T07:31:31.806102+0000","last_fullsized":"2024-09-19T07:31:31.806102+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:05.410032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53494,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":112,"num_read_kb":114,"num_write":156,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":160482,"data_compressed":5616,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247636,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351003,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1077248,"data_stored":553197,"data_compressed":26426,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28200,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716952,"statfs":{"total":95995035648,"available":95966158848,"internally_reserved":0,"allocated":1204224,"data_stored":1189956,"data_compressed":64146,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":557529,"data_compressed":26426,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005100,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":553353,"data_compressed":26426,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":1149657,"data_compressed":62274,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462966,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":548609,"data_compressed":26426,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724603,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":1144852,"data_compressed":62274,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888129,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":553652,"data_compressed":26426,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":548803,"data_compressed":26426,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":599360,"data_compressed":28298,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":597617,"data_compressed":28298,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":549997,"data_compressed":26426,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":49098,"data_compressed":1872,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":50841,"data_compressed":1872,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":51069,"data_compressed":1872,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:32:13.191 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T07:32:13.560 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:14.016 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:13 smithi159 ceph-mon[30472]: from='client.24755 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:32:14.017 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:13 smithi159 ceph-mon[30472]: pgmap v349: 129 pgs: 129 active+clean; 637 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-09-19T07:32:14.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:13 smithi203 ceph-mon[35335]: from='client.24755 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:32:14.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:13 smithi203 ceph-mon[35335]: pgmap v349: 129 pgs: 129 active+clean; 637 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-09-19T07:32:14.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:13 smithi189 ceph-mon[34851]: from='client.24755 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:32:14.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:13 smithi189 ceph-mon[34851]: pgmap v349: 129 pgs: 129 active+clean; 637 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-09-19T07:32:14.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.128+0000 7fd889200640 1 -- 172.21.15.159:0/2059419631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd884102780 msgr2=0x7fd8841067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:14.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.128+0000 7fd889200640 1 --2- 172.21.15.159:0/2059419631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd884102780 0x7fd8841067d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd870009900 tx=0x7fd87002a410 comp rx=0 tx=0).stop 2024-09-19T07:32:14.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.128+0000 7fd889200640 1 -- 172.21.15.159:0/2059419631 shutdown_connections 2024-09-19T07:32:14.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.128+0000 7fd889200640 1 --2- 172.21.15.159:0/2059419631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd884106f00 0x7fd88410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:14.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.128+0000 7fd889200640 1 --2- 172.21.15.159:0/2059419631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd884102780 0x7fd8841067d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:14.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.128+0000 7fd889200640 1 --2- 172.21.15.159:0/2059419631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd884101dd0 0x7fd8841021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:14.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.128+0000 7fd889200640 1 -- 172.21.15.159:0/2059419631 >> 172.21.15.159:0/2059419631 conn(0x7fd8840fd8b0 msgr2=0x7fd8840ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:14.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.128+0000 7fd889200640 1 -- 172.21.15.159:0/2059419631 shutdown_connections 2024-09-19T07:32:14.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.128+0000 7fd889200640 1 -- 172.21.15.159:0/2059419631 wait complete. 2024-09-19T07:32:14.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.129+0000 7fd889200640 1 Processor -- start 2024-09-19T07:32:14.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.129+0000 7fd889200640 1 -- start start 2024-09-19T07:32:14.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.130+0000 7fd889200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd884101dd0 0x7fd884198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:14.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.130+0000 7fd889200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd884102780 0x7fd884198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:14.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.130+0000 7fd889200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd884106f00 0x7fd88419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:14.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.130+0000 7fd889200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd884111f60 con 0x7fd884101dd0 2024-09-19T07:32:14.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.130+0000 7fd889200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd884111de0 con 0x7fd884102780 2024-09-19T07:32:14.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.130+0000 7fd889200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd8841120e0 con 0x7fd884106f00 2024-09-19T07:32:14.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.130+0000 7fd881e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd884101dd0 0x7fd884198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:14.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.130+0000 7fd881e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd884101dd0 0x7fd884198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38636/0 (socket says 172.21.15.159:38636) 2024-09-19T07:32:14.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.130+0000 7fd881e00640 1 -- 172.21.15.159:0/3038097649 learned_addr learned my addr 172.21.15.159:0/3038097649 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:14.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.130+0000 7fd882800640 1 --2- 172.21.15.159:0/3038097649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd884106f00 0x7fd88419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:14.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.130+0000 7fd881400640 1 --2- 172.21.15.159:0/3038097649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd884102780 0x7fd884198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:14.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.131+0000 7fd881e00640 1 -- 172.21.15.159:0/3038097649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd884106f00 msgr2=0x7fd88419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:14.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.131+0000 7fd881e00640 1 --2- 172.21.15.159:0/3038097649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd884106f00 0x7fd88419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:14.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.131+0000 7fd881e00640 1 -- 172.21.15.159:0/3038097649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd884102780 msgr2=0x7fd884198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:14.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.131+0000 7fd881e00640 1 --2- 172.21.15.159:0/3038097649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd884102780 0x7fd884198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:14.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.131+0000 7fd881e00640 1 -- 172.21.15.159:0/3038097649 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88419d3f0 con 0x7fd884101dd0 2024-09-19T07:32:14.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.131+0000 7fd881e00640 1 --2- 172.21.15.159:0/3038097649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd884101dd0 0x7fd884198440 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd87400e9f0 tx=0x7fd87400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:14.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.131+0000 7fd87aa00640 1 -- 172.21.15.159:0/3038097649 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87400cd80 con 0x7fd884101dd0 2024-09-19T07:32:14.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.131+0000 7fd87aa00640 1 -- 172.21.15.159:0/3038097649 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd874010910 con 0x7fd884101dd0 2024-09-19T07:32:14.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.131+0000 7fd889200640 1 -- 172.21.15.159:0/3038097649 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88419d6e0 con 0x7fd884101dd0 2024-09-19T07:32:14.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.131+0000 7fd87aa00640 1 -- 172.21.15.159:0/3038097649 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874018990 con 0x7fd884101dd0 2024-09-19T07:32:14.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.131+0000 7fd889200640 1 -- 172.21.15.159:0/3038097649 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8841a4fc0 con 0x7fd884101dd0 2024-09-19T07:32:14.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.133+0000 7fd889200640 1 -- 172.21.15.159:0/3038097649 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8841021b0 con 0x7fd884101dd0 2024-09-19T07:32:14.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.135+0000 7fd87aa00640 1 -- 172.21.15.159:0/3038097649 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd874010ac0 con 0x7fd884101dd0 2024-09-19T07:32:14.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.136+0000 7fd87aa00640 1 --2- 172.21.15.159:0/3038097649 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd8680a32d0 0x7fd8680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:14.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.136+0000 7fd87aa00640 1 -- 172.21.15.159:0/3038097649 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fd8740d03f0 con 0x7fd884101dd0 2024-09-19T07:32:14.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.136+0000 7fd881400640 1 --2- 172.21.15.159:0/3038097649 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd8680a32d0 0x7fd8680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:14.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.137+0000 7fd881400640 1 --2- 172.21.15.159:0/3038097649 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd8680a32d0 0x7fd8680a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd870009230 tx=0x7fd8700058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:14.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.140+0000 7fd87aa00640 1 -- 172.21.15.159:0/3038097649 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd87409cbf0 con 0x7fd884101dd0 2024-09-19T07:32:14.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.314+0000 7fd889200640 1 -- 172.21.15.159:0/3038097649 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd884062630 con 0x7fd8680a32d0 2024-09-19T07:32:14.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.322+0000 7fd87aa00640 1 -- 172.21.15.159:0/3038097649 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340651 (secure 0 0 0) 0x7fd884062630 con 0x7fd8680a32d0 2024-09-19T07:32:14.327 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:32:14.329 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-19T07:32:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.327+0000 7fd889200640 1 -- 172.21.15.159:0/3038097649 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd8680a32d0 msgr2=0x7fd8680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.327+0000 7fd889200640 1 --2- 172.21.15.159:0/3038097649 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd8680a32d0 0x7fd8680a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd870009230 tx=0x7fd8700058d0 comp rx=0 tx=0).stop 2024-09-19T07:32:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.327+0000 7fd889200640 1 -- 172.21.15.159:0/3038097649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd884101dd0 msgr2=0x7fd884198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.327+0000 7fd889200640 1 --2- 172.21.15.159:0/3038097649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd884101dd0 0x7fd884198440 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd87400e9f0 tx=0x7fd87400eeb0 comp rx=0 tx=0).stop 2024-09-19T07:32:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.327+0000 7fd889200640 1 -- 172.21.15.159:0/3038097649 shutdown_connections 2024-09-19T07:32:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.327+0000 7fd889200640 1 --2- 172.21.15.159:0/3038097649 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd8680a32d0 0x7fd8680a5790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.327+0000 7fd889200640 1 --2- 172.21.15.159:0/3038097649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd884106f00 0x7fd88419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.327+0000 7fd889200640 1 --2- 172.21.15.159:0/3038097649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd884102780 0x7fd884198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.327+0000 7fd889200640 1 --2- 172.21.15.159:0/3038097649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd884101dd0 0x7fd884198440 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.327+0000 7fd889200640 1 -- 172.21.15.159:0/3038097649 >> 172.21.15.159:0/3038097649 conn(0x7fd8840fd8b0 msgr2=0x7fd8840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:14.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.328+0000 7fd889200640 1 -- 172.21.15.159:0/3038097649 shutdown_connections 2024-09-19T07:32:14.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:14.328+0000 7fd889200640 1 -- 172.21.15.159:0/3038097649 wait complete. 2024-09-19T07:32:15.278 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-19T07:32:12.707420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652256,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":269,"num_read_kb":246,"num_write":342,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":354000,"kb_used_data":13072,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124587824,"statfs":{"total":1151940427776,"available":1151577931776,"internally_reserved":0,"allocated":13385728,"data_stored":8554742,"data_compressed":430272,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":54434,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":100,"num_write":30,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003349"},"pg_stats":[{"pgid":"2.1f","version":"98'9","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:50.896695+0000","last_change":"2024-09-19T07:30:59.408103+0000","last_active":"2024-09-19T07:31:50.896695+0000","last_peered":"2024-09-19T07:31:50.896695+0000","last_clean":"2024-09-19T07:31:50.896695+0000","last_became_active":"2024-09-19T07:30:59.407585+0000","last_became_peered":"2024-09-19T07:30:59.407585+0000","last_unstale":"2024-09-19T07:31:50.896695+0000","last_undegraded":"2024-09-19T07:31:50.896695+0000","last_fullsized":"2024-09-19T07:31:50.896695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:03.104939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806455+0000","last_change":"2024-09-19T07:31:02.438518+0000","last_active":"2024-09-19T07:31:31.806455+0000","last_peered":"2024-09-19T07:31:31.806455+0000","last_clean":"2024-09-19T07:31:31.806455+0000","last_became_active":"2024-09-19T07:31:02.438025+0000","last_became_peered":"2024-09-19T07:31:02.438025+0000","last_unstale":"2024-09-19T07:31:31.806455+0000","last_undegraded":"2024-09-19T07:31:31.806455+0000","last_fullsized":"2024-09-19T07:31:31.806455+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:59.148172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803301+0000","last_change":"2024-09-19T07:31:12.503517+0000","last_active":"2024-09-19T07:31:31.803301+0000","last_peered":"2024-09-19T07:31:31.803301+0000","last_clean":"2024-09-19T07:31:31.803301+0000","last_became_active":"2024-09-19T07:31:12.503155+0000","last_became_peered":"2024-09-19T07:31:12.503155+0000","last_unstale":"2024-09-19T07:31:31.803301+0000","last_undegraded":"2024-09-19T07:31:31.803301+0000","last_fullsized":"2024-09-19T07:31:31.803301+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:15.475388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806651+0000","last_change":"2024-09-19T07:30:59.445107+0000","last_active":"2024-09-19T07:31:31.806651+0000","last_peered":"2024-09-19T07:31:31.806651+0000","last_clean":"2024-09-19T07:31:31.806651+0000","last_became_active":"2024-09-19T07:30:59.444782+0000","last_became_peered":"2024-09-19T07:30:59.444782+0000","last_unstale":"2024-09-19T07:31:31.806651+0000","last_undegraded":"2024-09-19T07:31:31.806651+0000","last_fullsized":"2024-09-19T07:31:31.806651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:37.582314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.020219+0000","last_change":"2024-09-19T07:31:02.437690+0000","last_active":"2024-09-19T07:31:32.020219+0000","last_peered":"2024-09-19T07:31:32.020219+0000","last_clean":"2024-09-19T07:31:32.020219+0000","last_became_active":"2024-09-19T07:31:02.429969+0000","last_became_peered":"2024-09-19T07:31:02.429969+0000","last_unstale":"2024-09-19T07:31:32.020219+0000","last_undegraded":"2024-09-19T07:31:32.020219+0000","last_fullsized":"2024-09-19T07:31:32.020219+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:23.227114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805966+0000","last_change":"2024-09-19T07:31:12.500314+0000","last_active":"2024-09-19T07:31:31.805966+0000","last_peered":"2024-09-19T07:31:31.805966+0000","last_clean":"2024-09-19T07:31:31.805966+0000","last_became_active":"2024-09-19T07:31:12.499529+0000","last_became_peered":"2024-09-19T07:31:12.499529+0000","last_unstale":"2024-09-19T07:31:31.805966+0000","last_undegraded":"2024-09-19T07:31:31.805966+0000","last_fullsized":"2024-09-19T07:31:31.805966+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:46.805336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:50.895345+0000","last_change":"2024-09-19T07:30:59.408974+0000","last_active":"2024-09-19T07:31:50.895345+0000","last_peered":"2024-09-19T07:31:50.895345+0000","last_clean":"2024-09-19T07:31:50.895345+0000","last_became_active":"2024-09-19T07:30:59.408606+0000","last_became_peered":"2024-09-19T07:30:59.408606+0000","last_unstale":"2024-09-19T07:31:50.895345+0000","last_undegraded":"2024-09-19T07:31:50.895345+0000","last_fullsized":"2024-09-19T07:31:50.895345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:21.962123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.245608+0000","last_change":"2024-09-19T07:31:02.431202+0000","last_active":"2024-09-19T07:31:32.245608+0000","last_peered":"2024-09-19T07:31:32.245608+0000","last_clean":"2024-09-19T07:31:32.245608+0000","last_became_active":"2024-09-19T07:31:02.430789+0000","last_became_peered":"2024-09-19T07:31:02.430789+0000","last_unstale":"2024-09-19T07:31:32.245608+0000","last_undegraded":"2024-09-19T07:31:32.245608+0000","last_fullsized":"2024-09-19T07:31:32.245608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:40.750108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.807909+0000","last_change":"2024-09-19T07:31:12.500000+0000","last_active":"2024-09-19T07:31:31.807909+0000","last_peered":"2024-09-19T07:31:31.807909+0000","last_clean":"2024-09-19T07:31:31.807909+0000","last_became_active":"2024-09-19T07:31:12.499526+0000","last_became_peered":"2024-09-19T07:31:12.499526+0000","last_unstale":"2024-09-19T07:31:31.807909+0000","last_undegraded":"2024-09-19T07:31:31.807909+0000","last_fullsized":"2024-09-19T07:31:31.807909+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:15.449391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.246964+0000","last_change":"2024-09-19T07:30:59.411645+0000","last_active":"2024-09-19T07:31:32.246964+0000","last_peered":"2024-09-19T07:31:32.246964+0000","last_clean":"2024-09-19T07:31:32.246964+0000","last_became_active":"2024-09-19T07:30:59.411262+0000","last_became_peered":"2024-09-19T07:30:59.411262+0000","last_unstale":"2024-09-19T07:31:32.246964+0000","last_undegraded":"2024-09-19T07:31:32.246964+0000","last_fullsized":"2024-09-19T07:31:32.246964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:53.538203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.247958+0000","last_change":"2024-09-19T07:31:32.247958+0000","last_active":"2024-09-19T07:31:32.247958+0000","last_peered":"2024-09-19T07:31:32.247958+0000","last_clean":"2024-09-19T07:31:32.247958+0000","last_became_active":"2024-09-19T07:31:32.247521+0000","last_became_peered":"2024-09-19T07:31:32.247521+0000","last_unstale":"2024-09-19T07:31:32.247958+0000","last_undegraded":"2024-09-19T07:31:32.247958+0000","last_fullsized":"2024-09-19T07:31:32.247958+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:38.291915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806311+0000","last_change":"2024-09-19T07:31:12.500070+0000","last_active":"2024-09-19T07:31:31.806311+0000","last_peered":"2024-09-19T07:31:31.806311+0000","last_clean":"2024-09-19T07:31:31.806311+0000","last_became_active":"2024-09-19T07:31:12.499658+0000","last_became_peered":"2024-09-19T07:31:12.499658+0000","last_unstale":"2024-09-19T07:31:31.806311+0000","last_undegraded":"2024-09-19T07:31:31.806311+0000","last_fullsized":"2024-09-19T07:31:31.806311+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:08.955749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531082+0000","last_change":"2024-09-19T07:31:02.426972+0000","last_active":"2024-09-19T07:31:32.531082+0000","last_peered":"2024-09-19T07:31:32.531082+0000","last_clean":"2024-09-19T07:31:32.531082+0000","last_became_active":"2024-09-19T07:31:02.426661+0000","last_became_peered":"2024-09-19T07:31:02.426661+0000","last_unstale":"2024-09-19T07:31:32.531082+0000","last_undegraded":"2024-09-19T07:31:32.531082+0000","last_fullsized":"2024-09-19T07:31:32.531082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:26.943324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:49.886419+0000","last_change":"2024-09-19T07:30:59.410056+0000","last_active":"2024-09-19T07:31:49.886419+0000","last_peered":"2024-09-19T07:31:49.886419+0000","last_clean":"2024-09-19T07:31:49.886419+0000","last_became_active":"2024-09-19T07:30:59.409669+0000","last_became_peered":"2024-09-19T07:30:59.409669+0000","last_unstale":"2024-09-19T07:31:49.886419+0000","last_undegraded":"2024-09-19T07:31:49.886419+0000","last_fullsized":"2024-09-19T07:31:49.886419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:43.169905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806430+0000","last_change":"2024-09-19T07:31:12.557969+0000","last_active":"2024-09-19T07:31:31.806430+0000","last_peered":"2024-09-19T07:31:31.806430+0000","last_clean":"2024-09-19T07:31:31.806430+0000","last_became_active":"2024-09-19T07:31:12.556692+0000","last_became_peered":"2024-09-19T07:31:12.556692+0000","last_unstale":"2024-09-19T07:31:31.806430+0000","last_undegraded":"2024-09-19T07:31:31.806430+0000","last_fullsized":"2024-09-19T07:31:31.806430+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.563341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806099+0000","last_change":"2024-09-19T07:31:02.438979+0000","last_active":"2024-09-19T07:31:31.806099+0000","last_peered":"2024-09-19T07:31:31.806099+0000","last_clean":"2024-09-19T07:31:31.806099+0000","last_became_active":"2024-09-19T07:31:02.437192+0000","last_became_peered":"2024-09-19T07:31:02.437192+0000","last_unstale":"2024-09-19T07:31:31.806099+0000","last_undegraded":"2024-09-19T07:31:31.806099+0000","last_fullsized":"2024-09-19T07:31:31.806099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:39.957366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803751+0000","last_change":"2024-09-19T07:30:59.406970+0000","last_active":"2024-09-19T07:31:31.803751+0000","last_peered":"2024-09-19T07:31:31.803751+0000","last_clean":"2024-09-19T07:31:31.803751+0000","last_became_active":"2024-09-19T07:30:59.406537+0000","last_became_peered":"2024-09-19T07:30:59.406537+0000","last_unstale":"2024-09-19T07:31:31.803751+0000","last_undegraded":"2024-09-19T07:31:31.803751+0000","last_fullsized":"2024-09-19T07:31:31.803751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:32.969944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806387+0000","last_change":"2024-09-19T07:31:12.495709+0000","last_active":"2024-09-19T07:31:31.806387+0000","last_peered":"2024-09-19T07:31:31.806387+0000","last_clean":"2024-09-19T07:31:31.806387+0000","last_became_active":"2024-09-19T07:31:12.495201+0000","last_became_peered":"2024-09-19T07:31:12.495201+0000","last_unstale":"2024-09-19T07:31:31.806387+0000","last_undegraded":"2024-09-19T07:31:31.806387+0000","last_fullsized":"2024-09-19T07:31:31.806387+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:48.710011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.804813+0000","last_change":"2024-09-19T07:31:02.431032+0000","last_active":"2024-09-19T07:31:31.804813+0000","last_peered":"2024-09-19T07:31:31.804813+0000","last_clean":"2024-09-19T07:31:31.804813+0000","last_became_active":"2024-09-19T07:31:02.430691+0000","last_became_peered":"2024-09-19T07:31:02.430691+0000","last_unstale":"2024-09-19T07:31:31.804813+0000","last_undegraded":"2024-09-19T07:31:31.804813+0000","last_fullsized":"2024-09-19T07:31:31.804813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:59.627125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:49.887107+0000","last_change":"2024-09-19T07:30:59.410931+0000","last_active":"2024-09-19T07:31:49.887107+0000","last_peered":"2024-09-19T07:31:49.887107+0000","last_clean":"2024-09-19T07:31:49.887107+0000","last_became_active":"2024-09-19T07:30:59.410516+0000","last_became_peered":"2024-09-19T07:30:59.410516+0000","last_unstale":"2024-09-19T07:31:49.887107+0000","last_undegraded":"2024-09-19T07:31:49.887107+0000","last_fullsized":"2024-09-19T07:31:49.887107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:39.688441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.809669+0000","last_change":"2024-09-19T07:31:31.809224+0000","last_active":"2024-09-19T07:31:31.809669+0000","last_peered":"2024-09-19T07:31:31.809669+0000","last_clean":"2024-09-19T07:31:31.809669+0000","last_became_active":"2024-09-19T07:31:31.808673+0000","last_became_peered":"2024-09-19T07:31:31.808673+0000","last_unstale":"2024-09-19T07:31:31.809669+0000","last_undegraded":"2024-09-19T07:31:31.809669+0000","last_fullsized":"2024-09-19T07:31:31.809669+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:38.675403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.246222+0000","last_change":"2024-09-19T07:31:02.426735+0000","last_active":"2024-09-19T07:31:32.246222+0000","last_peered":"2024-09-19T07:31:32.246222+0000","last_clean":"2024-09-19T07:31:32.246222+0000","last_became_active":"2024-09-19T07:31:02.426324+0000","last_became_peered":"2024-09-19T07:31:02.426324+0000","last_unstale":"2024-09-19T07:31:32.246222+0000","last_undegraded":"2024-09-19T07:31:32.246222+0000","last_fullsized":"2024-09-19T07:31:32.246222+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:10.171191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:49.883039+0000","last_change":"2024-09-19T07:30:59.445125+0000","last_active":"2024-09-19T07:31:49.883039+0000","last_peered":"2024-09-19T07:31:49.883039+0000","last_clean":"2024-09-19T07:31:49.883039+0000","last_became_active":"2024-09-19T07:30:59.444757+0000","last_became_peered":"2024-09-19T07:30:59.444757+0000","last_unstale":"2024-09-19T07:31:49.883039+0000","last_undegraded":"2024-09-19T07:31:49.883039+0000","last_fullsized":"2024-09-19T07:31:49.883039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:34.183389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.807249+0000","last_change":"2024-09-19T07:31:12.557277+0000","last_active":"2024-09-19T07:31:31.807249+0000","last_peered":"2024-09-19T07:31:31.807249+0000","last_clean":"2024-09-19T07:31:31.807249+0000","last_became_active":"2024-09-19T07:31:12.556885+0000","last_became_peered":"2024-09-19T07:31:12.556885+0000","last_unstale":"2024-09-19T07:31:31.807249+0000","last_undegraded":"2024-09-19T07:31:31.807249+0000","last_fullsized":"2024-09-19T07:31:31.807249+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:39.960404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.020126+0000","last_change":"2024-09-19T07:31:02.441712+0000","last_active":"2024-09-19T07:31:32.020126+0000","last_peered":"2024-09-19T07:31:32.020126+0000","last_clean":"2024-09-19T07:31:32.020126+0000","last_became_active":"2024-09-19T07:31:02.440039+0000","last_became_peered":"2024-09-19T07:31:02.440039+0000","last_unstale":"2024-09-19T07:31:32.020126+0000","last_undegraded":"2024-09-19T07:31:32.020126+0000","last_fullsized":"2024-09-19T07:31:32.020126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:13.601774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:49.886162+0000","last_change":"2024-09-19T07:31:00.027265+0000","last_active":"2024-09-19T07:31:49.886162+0000","last_peered":"2024-09-19T07:31:49.886162+0000","last_clean":"2024-09-19T07:31:49.886162+0000","last_became_active":"2024-09-19T07:31:00.026924+0000","last_became_peered":"2024-09-19T07:31:00.026924+0000","last_unstale":"2024-09-19T07:31:49.886162+0000","last_undegraded":"2024-09-19T07:31:49.886162+0000","last_fullsized":"2024-09-19T07:31:49.886162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:05.290978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803449+0000","last_change":"2024-09-19T07:31:12.495613+0000","last_active":"2024-09-19T07:31:31.803449+0000","last_peered":"2024-09-19T07:31:31.803449+0000","last_clean":"2024-09-19T07:31:31.803449+0000","last_became_active":"2024-09-19T07:31:12.495218+0000","last_became_peered":"2024-09-19T07:31:12.495218+0000","last_unstale":"2024-09-19T07:31:31.803449+0000","last_undegraded":"2024-09-19T07:31:31.803449+0000","last_fullsized":"2024-09-19T07:31:31.803449+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:27.651671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806435+0000","last_change":"2024-09-19T07:31:02.442902+0000","last_active":"2024-09-19T07:31:31.806435+0000","last_peered":"2024-09-19T07:31:31.806435+0000","last_clean":"2024-09-19T07:31:31.806435+0000","last_became_active":"2024-09-19T07:31:02.437173+0000","last_became_peered":"2024-09-19T07:31:02.437173+0000","last_unstale":"2024-09-19T07:31:31.806435+0000","last_undegraded":"2024-09-19T07:31:31.806435+0000","last_fullsized":"2024-09-19T07:31:31.806435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:40:44.760641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'8","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:32:05.443697+0000","last_change":"2024-09-19T07:30:59.406488+0000","last_active":"2024-09-19T07:32:05.443697+0000","last_peered":"2024-09-19T07:32:05.443697+0000","last_clean":"2024-09-19T07:32:05.443697+0000","last_became_active":"2024-09-19T07:30:59.406121+0000","last_became_peered":"2024-09-19T07:30:59.406121+0000","last_unstale":"2024-09-19T07:32:05.443697+0000","last_undegraded":"2024-09-19T07:32:05.443697+0000","last_fullsized":"2024-09-19T07:32:05.443697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:14:50.534009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805273+0000","last_change":"2024-09-19T07:31:12.558127+0000","last_active":"2024-09-19T07:31:31.805273+0000","last_peered":"2024-09-19T07:31:31.805273+0000","last_clean":"2024-09-19T07:31:31.805273+0000","last_became_active":"2024-09-19T07:31:12.557078+0000","last_became_peered":"2024-09-19T07:31:12.557078+0000","last_unstale":"2024-09-19T07:31:31.805273+0000","last_undegraded":"2024-09-19T07:31:31.805273+0000","last_fullsized":"2024-09-19T07:31:31.805273+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:23.076116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803172+0000","last_change":"2024-09-19T07:31:02.433918+0000","last_active":"2024-09-19T07:31:31.803172+0000","last_peered":"2024-09-19T07:31:31.803172+0000","last_clean":"2024-09-19T07:31:31.803172+0000","last_became_active":"2024-09-19T07:31:02.430802+0000","last_became_peered":"2024-09-19T07:31:02.430802+0000","last_unstale":"2024-09-19T07:31:31.803172+0000","last_undegraded":"2024-09-19T07:31:31.803172+0000","last_fullsized":"2024-09-19T07:31:31.803172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:28.948441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:49.886123+0000","last_change":"2024-09-19T07:30:59.444178+0000","last_active":"2024-09-19T07:31:49.886123+0000","last_peered":"2024-09-19T07:31:49.886123+0000","last_clean":"2024-09-19T07:31:49.886123+0000","last_became_active":"2024-09-19T07:30:59.443847+0000","last_became_peered":"2024-09-19T07:30:59.443847+0000","last_unstale":"2024-09-19T07:31:49.886123+0000","last_undegraded":"2024-09-19T07:31:49.886123+0000","last_fullsized":"2024-09-19T07:31:49.886123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:08.588244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805143+0000","last_change":"2024-09-19T07:31:12.508847+0000","last_active":"2024-09-19T07:31:31.805143+0000","last_peered":"2024-09-19T07:31:31.805143+0000","last_clean":"2024-09-19T07:31:31.805143+0000","last_became_active":"2024-09-19T07:31:12.508371+0000","last_became_peered":"2024-09-19T07:31:12.508371+0000","last_unstale":"2024-09-19T07:31:31.805143+0000","last_undegraded":"2024-09-19T07:31:31.805143+0000","last_fullsized":"2024-09-19T07:31:31.805143+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:27.420724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805784+0000","last_change":"2024-09-19T07:31:02.438213+0000","last_active":"2024-09-19T07:31:31.805784+0000","last_peered":"2024-09-19T07:31:31.805784+0000","last_clean":"2024-09-19T07:31:31.805784+0000","last_became_active":"2024-09-19T07:31:02.437892+0000","last_became_peered":"2024-09-19T07:31:02.437892+0000","last_unstale":"2024-09-19T07:31:31.805784+0000","last_undegraded":"2024-09-19T07:31:31.805784+0000","last_fullsized":"2024-09-19T07:31:31.805784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:50.249962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.809028+0000","last_change":"2024-09-19T07:30:59.445523+0000","last_active":"2024-09-19T07:31:31.809028+0000","last_peered":"2024-09-19T07:31:31.809028+0000","last_clean":"2024-09-19T07:31:31.809028+0000","last_became_active":"2024-09-19T07:30:59.445118+0000","last_became_peered":"2024-09-19T07:30:59.445118+0000","last_unstale":"2024-09-19T07:31:31.809028+0000","last_undegraded":"2024-09-19T07:31:31.809028+0000","last_fullsized":"2024-09-19T07:31:31.809028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:35.369102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.246349+0000","last_change":"2024-09-19T07:31:12.557541+0000","last_active":"2024-09-19T07:31:32.246349+0000","last_peered":"2024-09-19T07:31:32.246349+0000","last_clean":"2024-09-19T07:31:32.246349+0000","last_became_active":"2024-09-19T07:31:12.557130+0000","last_became_peered":"2024-09-19T07:31:12.557130+0000","last_unstale":"2024-09-19T07:31:32.246349+0000","last_undegraded":"2024-09-19T07:31:32.246349+0000","last_fullsized":"2024-09-19T07:31:32.246349+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:52.610222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805541+0000","last_change":"2024-09-19T07:31:02.434743+0000","last_active":"2024-09-19T07:31:31.805541+0000","last_peered":"2024-09-19T07:31:31.805541+0000","last_clean":"2024-09-19T07:31:31.805541+0000","last_became_active":"2024-09-19T07:31:02.434312+0000","last_became_peered":"2024-09-19T07:31:02.434312+0000","last_unstale":"2024-09-19T07:31:31.805541+0000","last_undegraded":"2024-09-19T07:31:31.805541+0000","last_fullsized":"2024-09-19T07:31:31.805541+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:24:00.569195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531239+0000","last_change":"2024-09-19T07:30:59.409893+0000","last_active":"2024-09-19T07:31:32.531239+0000","last_peered":"2024-09-19T07:31:32.531239+0000","last_clean":"2024-09-19T07:31:32.531239+0000","last_became_active":"2024-09-19T07:30:59.409440+0000","last_became_peered":"2024-09-19T07:30:59.409440+0000","last_unstale":"2024-09-19T07:31:32.531239+0000","last_undegraded":"2024-09-19T07:31:32.531239+0000","last_fullsized":"2024-09-19T07:31:32.531239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:45.999596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803030+0000","last_change":"2024-09-19T07:31:12.505965+0000","last_active":"2024-09-19T07:31:31.803030+0000","last_peered":"2024-09-19T07:31:31.803030+0000","last_clean":"2024-09-19T07:31:31.803030+0000","last_became_active":"2024-09-19T07:31:12.504675+0000","last_became_peered":"2024-09-19T07:31:12.504675+0000","last_unstale":"2024-09-19T07:31:31.803030+0000","last_undegraded":"2024-09-19T07:31:31.803030+0000","last_fullsized":"2024-09-19T07:31:31.803030+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:47.345020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.809488+0000","last_change":"2024-09-19T07:31:31.809488+0000","last_active":"2024-09-19T07:31:31.809488+0000","last_peered":"2024-09-19T07:31:31.809488+0000","last_clean":"2024-09-19T07:31:31.809488+0000","last_became_active":"2024-09-19T07:31:31.809107+0000","last_became_peered":"2024-09-19T07:31:31.809107+0000","last_unstale":"2024-09-19T07:31:31.809488+0000","last_undegraded":"2024-09-19T07:31:31.809488+0000","last_fullsized":"2024-09-19T07:31:31.809488+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:39.757841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":65,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:32:10.924747+0000","last_change":"2024-09-19T07:30:59.445001+0000","last_active":"2024-09-19T07:32:10.924747+0000","last_peered":"2024-09-19T07:32:10.924747+0000","last_clean":"2024-09-19T07:32:10.924747+0000","last_became_active":"2024-09-19T07:30:59.444590+0000","last_became_peered":"2024-09-19T07:30:59.444590+0000","last_unstale":"2024-09-19T07:32:10.924747+0000","last_undegraded":"2024-09-19T07:32:10.924747+0000","last_fullsized":"2024-09-19T07:32:10.924747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:13.934085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803248+0000","last_change":"2024-09-19T07:31:12.557622+0000","last_active":"2024-09-19T07:31:31.803248+0000","last_peered":"2024-09-19T07:31:31.803248+0000","last_clean":"2024-09-19T07:31:31.803248+0000","last_became_active":"2024-09-19T07:31:12.556934+0000","last_became_peered":"2024-09-19T07:31:12.556934+0000","last_unstale":"2024-09-19T07:31:31.803248+0000","last_undegraded":"2024-09-19T07:31:31.803248+0000","last_fullsized":"2024-09-19T07:31:31.803248+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:06:31.944819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.802694+0000","last_change":"2024-09-19T07:31:02.433199+0000","last_active":"2024-09-19T07:31:31.802694+0000","last_peered":"2024-09-19T07:31:31.802694+0000","last_clean":"2024-09-19T07:31:31.802694+0000","last_became_active":"2024-09-19T07:31:02.432739+0000","last_became_peered":"2024-09-19T07:31:02.432739+0000","last_unstale":"2024-09-19T07:31:31.802694+0000","last_undegraded":"2024-09-19T07:31:31.802694+0000","last_fullsized":"2024-09-19T07:31:31.802694+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:57.392885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:50.894688+0000","last_change":"2024-09-19T07:30:59.413041+0000","last_active":"2024-09-19T07:31:50.894688+0000","last_peered":"2024-09-19T07:31:50.894688+0000","last_clean":"2024-09-19T07:31:50.894688+0000","last_became_active":"2024-09-19T07:30:59.412483+0000","last_became_peered":"2024-09-19T07:30:59.412483+0000","last_unstale":"2024-09-19T07:31:50.894688+0000","last_undegraded":"2024-09-19T07:31:50.894688+0000","last_fullsized":"2024-09-19T07:31:50.894688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:05.151006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806998+0000","last_change":"2024-09-19T07:31:12.557638+0000","last_active":"2024-09-19T07:31:31.806998+0000","last_peered":"2024-09-19T07:31:31.806998+0000","last_clean":"2024-09-19T07:31:31.806998+0000","last_became_active":"2024-09-19T07:31:12.557170+0000","last_became_peered":"2024-09-19T07:31:12.557170+0000","last_unstale":"2024-09-19T07:31:31.806998+0000","last_undegraded":"2024-09-19T07:31:31.806998+0000","last_fullsized":"2024-09-19T07:31:31.806998+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:14.077038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806268+0000","last_change":"2024-09-19T07:31:02.443081+0000","last_active":"2024-09-19T07:31:31.806268+0000","last_peered":"2024-09-19T07:31:31.806268+0000","last_clean":"2024-09-19T07:31:31.806268+0000","last_became_active":"2024-09-19T07:31:02.437034+0000","last_became_peered":"2024-09-19T07:31:02.437034+0000","last_unstale":"2024-09-19T07:31:31.806268+0000","last_undegraded":"2024-09-19T07:31:31.806268+0000","last_fullsized":"2024-09-19T07:31:31.806268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:55.310844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:51.895562+0000","last_change":"2024-09-19T07:30:59.410069+0000","last_active":"2024-09-19T07:31:51.895562+0000","last_peered":"2024-09-19T07:31:51.895562+0000","last_clean":"2024-09-19T07:31:51.895562+0000","last_became_active":"2024-09-19T07:30:59.409736+0000","last_became_peered":"2024-09-19T07:30:59.409736+0000","last_unstale":"2024-09-19T07:31:51.895562+0000","last_undegraded":"2024-09-19T07:31:51.895562+0000","last_fullsized":"2024-09-19T07:31:51.895562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:59:47.095227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":356,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531185+0000","last_change":"2024-09-19T07:29:38.994670+0000","last_active":"2024-09-19T07:31:32.531185+0000","last_peered":"2024-09-19T07:31:32.531185+0000","last_clean":"2024-09-19T07:31:32.531185+0000","last_became_active":"2024-09-19T07:29:38.994229+0000","last_became_peered":"2024-09-19T07:29:38.994229+0000","last_unstale":"2024-09-19T07:31:32.531185+0000","last_undegraded":"2024-09-19T07:31:32.531185+0000","last_fullsized":"2024-09-19T07:31:32.531185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:25:55.396577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:25:55.396577+0000","last_clean_scrub_stamp":"2024-09-19T07:25:55.396577+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:06.972254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531547+0000","last_change":"2024-09-19T07:31:12.497060+0000","last_active":"2024-09-19T07:31:32.531547+0000","last_peered":"2024-09-19T07:31:32.531547+0000","last_clean":"2024-09-19T07:31:32.531547+0000","last_became_active":"2024-09-19T07:31:12.496693+0000","last_became_peered":"2024-09-19T07:31:12.496693+0000","last_unstale":"2024-09-19T07:31:32.531547+0000","last_undegraded":"2024-09-19T07:31:32.531547+0000","last_fullsized":"2024-09-19T07:31:32.531547+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:41.879585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531603+0000","last_change":"2024-09-19T07:31:02.420732+0000","last_active":"2024-09-19T07:31:32.531603+0000","last_peered":"2024-09-19T07:31:32.531603+0000","last_clean":"2024-09-19T07:31:32.531603+0000","last_became_active":"2024-09-19T07:31:02.420394+0000","last_became_peered":"2024-09-19T07:31:02.420394+0000","last_unstale":"2024-09-19T07:31:32.531603+0000","last_undegraded":"2024-09-19T07:31:32.531603+0000","last_fullsized":"2024-09-19T07:31:32.531603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:48:36.157189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:50.896898+0000","last_change":"2024-09-19T07:31:00.027344+0000","last_active":"2024-09-19T07:31:50.896898+0000","last_peered":"2024-09-19T07:31:50.896898+0000","last_clean":"2024-09-19T07:31:50.896898+0000","last_became_active":"2024-09-19T07:31:00.026855+0000","last_became_peered":"2024-09-19T07:31:00.026855+0000","last_unstale":"2024-09-19T07:31:50.896898+0000","last_undegraded":"2024-09-19T07:31:50.896898+0000","last_fullsized":"2024-09-19T07:31:50.896898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:12.304161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.245452+0000","last_change":"2024-09-19T07:31:12.495883+0000","last_active":"2024-09-19T07:31:32.245452+0000","last_peered":"2024-09-19T07:31:32.245452+0000","last_clean":"2024-09-19T07:31:32.245452+0000","last_became_active":"2024-09-19T07:31:12.495253+0000","last_became_peered":"2024-09-19T07:31:12.495253+0000","last_unstale":"2024-09-19T07:31:32.245452+0000","last_undegraded":"2024-09-19T07:31:32.245452+0000","last_fullsized":"2024-09-19T07:31:32.245452+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:56.608977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531431+0000","last_change":"2024-09-19T07:31:02.427706+0000","last_active":"2024-09-19T07:31:32.531431+0000","last_peered":"2024-09-19T07:31:32.531431+0000","last_clean":"2024-09-19T07:31:32.531431+0000","last_became_active":"2024-09-19T07:31:02.427398+0000","last_became_peered":"2024-09-19T07:31:02.427398+0000","last_unstale":"2024-09-19T07:31:32.531431+0000","last_undegraded":"2024-09-19T07:31:32.531431+0000","last_fullsized":"2024-09-19T07:31:32.531431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:13.179472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.807839+0000","last_change":"2024-09-19T07:30:59.410020+0000","last_active":"2024-09-19T07:31:31.807839+0000","last_peered":"2024-09-19T07:31:31.807839+0000","last_clean":"2024-09-19T07:31:31.807839+0000","last_became_active":"2024-09-19T07:30:59.409633+0000","last_became_peered":"2024-09-19T07:30:59.409633+0000","last_unstale":"2024-09-19T07:31:31.807839+0000","last_undegraded":"2024-09-19T07:31:31.807839+0000","last_fullsized":"2024-09-19T07:31:31.807839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:49.673952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.019882+0000","last_change":"2024-09-19T07:31:12.511259+0000","last_active":"2024-09-19T07:31:32.019882+0000","last_peered":"2024-09-19T07:31:32.019882+0000","last_clean":"2024-09-19T07:31:32.019882+0000","last_became_active":"2024-09-19T07:31:12.510691+0000","last_became_peered":"2024-09-19T07:31:12.510691+0000","last_unstale":"2024-09-19T07:31:32.019882+0000","last_undegraded":"2024-09-19T07:31:32.019882+0000","last_fullsized":"2024-09-19T07:31:32.019882+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:09.388922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531697+0000","last_change":"2024-09-19T07:31:02.425922+0000","last_active":"2024-09-19T07:31:32.531697+0000","last_peered":"2024-09-19T07:31:32.531697+0000","last_clean":"2024-09-19T07:31:32.531697+0000","last_became_active":"2024-09-19T07:31:02.425565+0000","last_became_peered":"2024-09-19T07:31:02.425565+0000","last_unstale":"2024-09-19T07:31:32.531697+0000","last_undegraded":"2024-09-19T07:31:32.531697+0000","last_fullsized":"2024-09-19T07:31:32.531697+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:48.997362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":52,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.813529+0000","last_change":"2024-09-19T07:31:31.813529+0000","last_active":"2024-09-19T07:31:31.813529+0000","last_peered":"2024-09-19T07:31:31.813529+0000","last_clean":"2024-09-19T07:31:31.813529+0000","last_became_active":"2024-09-19T07:31:31.809205+0000","last_became_peered":"2024-09-19T07:31:31.809205+0000","last_unstale":"2024-09-19T07:31:31.813529+0000","last_undegraded":"2024-09-19T07:31:31.813529+0000","last_fullsized":"2024-09-19T07:31:31.813529+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:26.117905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806457+0000","last_change":"2024-09-19T07:31:12.497728+0000","last_active":"2024-09-19T07:31:31.806457+0000","last_peered":"2024-09-19T07:31:31.806457+0000","last_clean":"2024-09-19T07:31:31.806457+0000","last_became_active":"2024-09-19T07:31:12.497315+0000","last_became_peered":"2024-09-19T07:31:12.497315+0000","last_unstale":"2024-09-19T07:31:31.806457+0000","last_undegraded":"2024-09-19T07:31:31.806457+0000","last_fullsized":"2024-09-19T07:31:31.806457+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:12.174147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805094+0000","last_change":"2024-09-19T07:31:02.433205+0000","last_active":"2024-09-19T07:31:31.805094+0000","last_peered":"2024-09-19T07:31:31.805094+0000","last_clean":"2024-09-19T07:31:31.805094+0000","last_became_active":"2024-09-19T07:31:02.432856+0000","last_became_peered":"2024-09-19T07:31:02.432856+0000","last_unstale":"2024-09-19T07:31:31.805094+0000","last_undegraded":"2024-09-19T07:31:31.805094+0000","last_fullsized":"2024-09-19T07:31:31.805094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:31.470420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:51.890751+0000","last_change":"2024-09-19T07:30:59.411966+0000","last_active":"2024-09-19T07:31:51.890751+0000","last_peered":"2024-09-19T07:31:51.890751+0000","last_clean":"2024-09-19T07:31:51.890751+0000","last_became_active":"2024-09-19T07:30:59.411595+0000","last_became_peered":"2024-09-19T07:30:59.411595+0000","last_unstale":"2024-09-19T07:31:51.890751+0000","last_undegraded":"2024-09-19T07:31:51.890751+0000","last_fullsized":"2024-09-19T07:31:51.890751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:25.539059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806569+0000","last_change":"2024-09-19T07:31:12.500715+0000","last_active":"2024-09-19T07:31:31.806569+0000","last_peered":"2024-09-19T07:31:31.806569+0000","last_clean":"2024-09-19T07:31:31.806569+0000","last_became_active":"2024-09-19T07:31:12.500401+0000","last_became_peered":"2024-09-19T07:31:12.500401+0000","last_unstale":"2024-09-19T07:31:31.806569+0000","last_undegraded":"2024-09-19T07:31:31.806569+0000","last_fullsized":"2024-09-19T07:31:31.806569+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:59.158518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806740+0000","last_change":"2024-09-19T07:31:02.429062+0000","last_active":"2024-09-19T07:31:31.806740+0000","last_peered":"2024-09-19T07:31:31.806740+0000","last_clean":"2024-09-19T07:31:31.806740+0000","last_became_active":"2024-09-19T07:31:02.428703+0000","last_became_peered":"2024-09-19T07:31:02.428703+0000","last_unstale":"2024-09-19T07:31:31.806740+0000","last_undegraded":"2024-09-19T07:31:31.806740+0000","last_fullsized":"2024-09-19T07:31:31.806740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:30.928698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":55,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:50.897145+0000","last_change":"2024-09-19T07:30:59.444883+0000","last_active":"2024-09-19T07:31:50.897145+0000","last_peered":"2024-09-19T07:31:50.897145+0000","last_clean":"2024-09-19T07:31:50.897145+0000","last_became_active":"2024-09-19T07:30:59.444445+0000","last_became_peered":"2024-09-19T07:30:59.444445+0000","last_unstale":"2024-09-19T07:31:50.897145+0000","last_undegraded":"2024-09-19T07:31:50.897145+0000","last_fullsized":"2024-09-19T07:31:50.897145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:29:11.166450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803238+0000","last_change":"2024-09-19T07:31:12.508431+0000","last_active":"2024-09-19T07:31:31.803238+0000","last_peered":"2024-09-19T07:31:31.803238+0000","last_clean":"2024-09-19T07:31:31.803238+0000","last_became_active":"2024-09-19T07:31:12.507826+0000","last_became_peered":"2024-09-19T07:31:12.507826+0000","last_unstale":"2024-09-19T07:31:31.803238+0000","last_undegraded":"2024-09-19T07:31:31.803238+0000","last_fullsized":"2024-09-19T07:31:31.803238+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:59.545418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.530878+0000","last_change":"2024-09-19T07:31:02.423973+0000","last_active":"2024-09-19T07:31:32.530878+0000","last_peered":"2024-09-19T07:31:32.530878+0000","last_clean":"2024-09-19T07:31:32.530878+0000","last_became_active":"2024-09-19T07:31:02.423609+0000","last_became_peered":"2024-09-19T07:31:02.423609+0000","last_unstale":"2024-09-19T07:31:32.530878+0000","last_undegraded":"2024-09-19T07:31:32.530878+0000","last_fullsized":"2024-09-19T07:31:32.530878+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:23:47.693286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"98'4","reported_seq":66,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:32:09.923915+0000","last_change":"2024-09-19T07:30:59.412862+0000","last_active":"2024-09-19T07:32:09.923915+0000","last_peered":"2024-09-19T07:32:09.923915+0000","last_clean":"2024-09-19T07:32:09.923915+0000","last_became_active":"2024-09-19T07:30:59.412401+0000","last_became_peered":"2024-09-19T07:30:59.412401+0000","last_unstale":"2024-09-19T07:32:09.923915+0000","last_undegraded":"2024-09-19T07:32:09.923915+0000","last_fullsized":"2024-09-19T07:32:09.923915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:49.461312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.808794+0000","last_change":"2024-09-19T07:31:12.507714+0000","last_active":"2024-09-19T07:31:31.808794+0000","last_peered":"2024-09-19T07:31:31.808794+0000","last_clean":"2024-09-19T07:31:31.808794+0000","last_became_active":"2024-09-19T07:31:12.506054+0000","last_became_peered":"2024-09-19T07:31:12.506054+0000","last_unstale":"2024-09-19T07:31:31.808794+0000","last_undegraded":"2024-09-19T07:31:31.808794+0000","last_fullsized":"2024-09-19T07:31:31.808794+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:18.070858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531335+0000","last_change":"2024-09-19T07:31:02.424718+0000","last_active":"2024-09-19T07:31:32.531335+0000","last_peered":"2024-09-19T07:31:32.531335+0000","last_clean":"2024-09-19T07:31:32.531335+0000","last_became_active":"2024-09-19T07:31:02.424351+0000","last_became_peered":"2024-09-19T07:31:02.424351+0000","last_unstale":"2024-09-19T07:31:32.531335+0000","last_undegraded":"2024-09-19T07:31:32.531335+0000","last_fullsized":"2024-09-19T07:31:32.531335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:59.396752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:49.886069+0000","last_change":"2024-09-19T07:30:59.410551+0000","last_active":"2024-09-19T07:31:49.886069+0000","last_peered":"2024-09-19T07:31:49.886069+0000","last_clean":"2024-09-19T07:31:49.886069+0000","last_became_active":"2024-09-19T07:30:59.410226+0000","last_became_peered":"2024-09-19T07:30:59.410226+0000","last_unstale":"2024-09-19T07:31:49.886069+0000","last_undegraded":"2024-09-19T07:31:49.886069+0000","last_fullsized":"2024-09-19T07:31:49.886069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:18.815424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.532426+0000","last_change":"2024-09-19T07:31:32.532426+0000","last_active":"2024-09-19T07:31:32.532426+0000","last_peered":"2024-09-19T07:31:32.532426+0000","last_clean":"2024-09-19T07:31:32.532426+0000","last_became_active":"2024-09-19T07:31:32.532034+0000","last_became_peered":"2024-09-19T07:31:32.532034+0000","last_unstale":"2024-09-19T07:31:32.532426+0000","last_undegraded":"2024-09-19T07:31:32.532426+0000","last_fullsized":"2024-09-19T07:31:32.532426+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:09.421369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.802848+0000","last_change":"2024-09-19T07:31:02.444240+0000","last_active":"2024-09-19T07:31:31.802848+0000","last_peered":"2024-09-19T07:31:31.802848+0000","last_clean":"2024-09-19T07:31:31.802848+0000","last_became_active":"2024-09-19T07:31:02.443808+0000","last_became_peered":"2024-09-19T07:31:02.443808+0000","last_unstale":"2024-09-19T07:31:31.802848+0000","last_undegraded":"2024-09-19T07:31:31.802848+0000","last_fullsized":"2024-09-19T07:31:31.802848+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:12.898708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:50.896980+0000","last_change":"2024-09-19T07:30:59.411130+0000","last_active":"2024-09-19T07:31:50.896980+0000","last_peered":"2024-09-19T07:31:50.896980+0000","last_clean":"2024-09-19T07:31:50.896980+0000","last_became_active":"2024-09-19T07:30:59.410837+0000","last_became_peered":"2024-09-19T07:30:59.410837+0000","last_unstale":"2024-09-19T07:31:50.896980+0000","last_undegraded":"2024-09-19T07:31:50.896980+0000","last_fullsized":"2024-09-19T07:31:50.896980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:04.194868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.248768+0000","last_change":"2024-09-19T07:31:32.248185+0000","last_active":"2024-09-19T07:31:32.248768+0000","last_peered":"2024-09-19T07:31:32.248768+0000","last_clean":"2024-09-19T07:31:32.248768+0000","last_became_active":"2024-09-19T07:31:32.247514+0000","last_became_peered":"2024-09-19T07:31:32.247514+0000","last_unstale":"2024-09-19T07:31:32.248768+0000","last_undegraded":"2024-09-19T07:31:32.248768+0000","last_fullsized":"2024-09-19T07:31:32.248768+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:09.713374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.246695+0000","last_change":"2024-09-19T07:31:02.421038+0000","last_active":"2024-09-19T07:31:32.246695+0000","last_peered":"2024-09-19T07:31:32.246695+0000","last_clean":"2024-09-19T07:31:32.246695+0000","last_became_active":"2024-09-19T07:31:02.420715+0000","last_became_peered":"2024-09-19T07:31:02.420715+0000","last_unstale":"2024-09-19T07:31:32.246695+0000","last_undegraded":"2024-09-19T07:31:32.246695+0000","last_fullsized":"2024-09-19T07:31:32.246695+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:39.943123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"98'29","reported_seq":112,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:32:09.924374+0000","last_change":"2024-09-19T07:30:59.414095+0000","last_active":"2024-09-19T07:32:09.924374+0000","last_peered":"2024-09-19T07:32:09.924374+0000","last_clean":"2024-09-19T07:32:09.924374+0000","last_became_active":"2024-09-19T07:30:59.413431+0000","last_became_peered":"2024-09-19T07:30:59.413431+0000","last_unstale":"2024-09-19T07:32:09.924374+0000","last_undegraded":"2024-09-19T07:32:09.924374+0000","last_fullsized":"2024-09-19T07:32:09.924374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:00.573369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57258,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":77,"num_write":30,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.808429+0000","last_change":"2024-09-19T07:31:12.500563+0000","last_active":"2024-09-19T07:31:31.808429+0000","last_peered":"2024-09-19T07:31:31.808429+0000","last_clean":"2024-09-19T07:31:31.808429+0000","last_became_active":"2024-09-19T07:31:12.500170+0000","last_became_peered":"2024-09-19T07:31:12.500170+0000","last_unstale":"2024-09-19T07:31:31.808429+0000","last_undegraded":"2024-09-19T07:31:31.808429+0000","last_fullsized":"2024-09-19T07:31:31.808429+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:17.897055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.245660+0000","last_change":"2024-09-19T07:31:02.431931+0000","last_active":"2024-09-19T07:31:32.245660+0000","last_peered":"2024-09-19T07:31:32.245660+0000","last_clean":"2024-09-19T07:31:32.245660+0000","last_became_active":"2024-09-19T07:31:02.431579+0000","last_became_peered":"2024-09-19T07:31:02.431579+0000","last_unstale":"2024-09-19T07:31:32.245660+0000","last_undegraded":"2024-09-19T07:31:32.245660+0000","last_fullsized":"2024-09-19T07:31:32.245660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:18.260616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:49.885400+0000","last_change":"2024-09-19T07:30:59.445150+0000","last_active":"2024-09-19T07:31:49.885400+0000","last_peered":"2024-09-19T07:31:49.885400+0000","last_clean":"2024-09-19T07:31:49.885400+0000","last_became_active":"2024-09-19T07:30:59.444776+0000","last_became_peered":"2024-09-19T07:30:59.444776+0000","last_unstale":"2024-09-19T07:31:49.885400+0000","last_undegraded":"2024-09-19T07:31:49.885400+0000","last_fullsized":"2024-09-19T07:31:49.885400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:30.654652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.248246+0000","last_change":"2024-09-19T07:31:32.248246+0000","last_active":"2024-09-19T07:31:32.248246+0000","last_peered":"2024-09-19T07:31:32.248246+0000","last_clean":"2024-09-19T07:31:32.248246+0000","last_became_active":"2024-09-19T07:31:32.247456+0000","last_became_peered":"2024-09-19T07:31:32.247456+0000","last_unstale":"2024-09-19T07:31:32.248246+0000","last_undegraded":"2024-09-19T07:31:32.248246+0000","last_fullsized":"2024-09-19T07:31:32.248246+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:31:44.320026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.020209+0000","last_change":"2024-09-19T07:31:02.441924+0000","last_active":"2024-09-19T07:31:32.020209+0000","last_peered":"2024-09-19T07:31:32.020209+0000","last_clean":"2024-09-19T07:31:32.020209+0000","last_became_active":"2024-09-19T07:31:02.438247+0000","last_became_peered":"2024-09-19T07:31:02.438247+0000","last_unstale":"2024-09-19T07:31:32.020209+0000","last_undegraded":"2024-09-19T07:31:32.020209+0000","last_fullsized":"2024-09-19T07:31:32.020209+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:12.878185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:49.884781+0000","last_change":"2024-09-19T07:30:59.409679+0000","last_active":"2024-09-19T07:31:49.884781+0000","last_peered":"2024-09-19T07:31:49.884781+0000","last_clean":"2024-09-19T07:31:49.884781+0000","last_became_active":"2024-09-19T07:30:59.409313+0000","last_became_peered":"2024-09-19T07:30:59.409313+0000","last_unstale":"2024-09-19T07:31:49.884781+0000","last_undegraded":"2024-09-19T07:31:49.884781+0000","last_fullsized":"2024-09-19T07:31:49.884781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:42.121358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806947+0000","last_change":"2024-09-19T07:31:12.506596+0000","last_active":"2024-09-19T07:31:31.806947+0000","last_peered":"2024-09-19T07:31:31.806947+0000","last_clean":"2024-09-19T07:31:31.806947+0000","last_became_active":"2024-09-19T07:31:12.505107+0000","last_became_peered":"2024-09-19T07:31:12.505107+0000","last_unstale":"2024-09-19T07:31:31.806947+0000","last_undegraded":"2024-09-19T07:31:31.806947+0000","last_fullsized":"2024-09-19T07:31:31.806947+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:39.157988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.246037+0000","last_change":"2024-09-19T07:31:02.429883+0000","last_active":"2024-09-19T07:31:32.246037+0000","last_peered":"2024-09-19T07:31:32.246037+0000","last_clean":"2024-09-19T07:31:32.246037+0000","last_became_active":"2024-09-19T07:31:02.429554+0000","last_became_peered":"2024-09-19T07:31:02.429554+0000","last_unstale":"2024-09-19T07:31:32.246037+0000","last_undegraded":"2024-09-19T07:31:32.246037+0000","last_fullsized":"2024-09-19T07:31:32.246037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:00.336506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":61,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:32:08.922089+0000","last_change":"2024-09-19T07:30:59.414735+0000","last_active":"2024-09-19T07:32:08.922089+0000","last_peered":"2024-09-19T07:32:08.922089+0000","last_clean":"2024-09-19T07:32:08.922089+0000","last_became_active":"2024-09-19T07:30:59.414385+0000","last_became_peered":"2024-09-19T07:30:59.414385+0000","last_unstale":"2024-09-19T07:32:08.922089+0000","last_undegraded":"2024-09-19T07:32:08.922089+0000","last_fullsized":"2024-09-19T07:32:08.922089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:43.668794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805429+0000","last_change":"2024-09-19T07:31:12.497242+0000","last_active":"2024-09-19T07:31:31.805429+0000","last_peered":"2024-09-19T07:31:31.805429+0000","last_clean":"2024-09-19T07:31:31.805429+0000","last_became_active":"2024-09-19T07:31:12.496857+0000","last_became_peered":"2024-09-19T07:31:12.496857+0000","last_unstale":"2024-09-19T07:31:31.805429+0000","last_undegraded":"2024-09-19T07:31:31.805429+0000","last_fullsized":"2024-09-19T07:31:31.805429+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:18.418594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.020041+0000","last_change":"2024-09-19T07:31:12.504507+0000","last_active":"2024-09-19T07:31:32.020041+0000","last_peered":"2024-09-19T07:31:32.020041+0000","last_clean":"2024-09-19T07:31:32.020041+0000","last_became_active":"2024-09-19T07:31:12.503289+0000","last_became_peered":"2024-09-19T07:31:12.503289+0000","last_unstale":"2024-09-19T07:31:32.020041+0000","last_undegraded":"2024-09-19T07:31:32.020041+0000","last_fullsized":"2024-09-19T07:31:32.020041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:21.771692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.807713+0000","last_change":"2024-09-19T07:31:02.429634+0000","last_active":"2024-09-19T07:31:31.807713+0000","last_peered":"2024-09-19T07:31:31.807713+0000","last_clean":"2024-09-19T07:31:31.807713+0000","last_became_active":"2024-09-19T07:31:02.429280+0000","last_became_peered":"2024-09-19T07:31:02.429280+0000","last_unstale":"2024-09-19T07:31:31.807713+0000","last_undegraded":"2024-09-19T07:31:31.807713+0000","last_fullsized":"2024-09-19T07:31:31.807713+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:13.872594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.807160+0000","last_change":"2024-09-19T07:30:59.409961+0000","last_active":"2024-09-19T07:31:31.807160+0000","last_peered":"2024-09-19T07:31:31.807160+0000","last_clean":"2024-09-19T07:31:31.807160+0000","last_became_active":"2024-09-19T07:30:59.409582+0000","last_became_peered":"2024-09-19T07:30:59.409582+0000","last_unstale":"2024-09-19T07:31:31.807160+0000","last_undegraded":"2024-09-19T07:31:31.807160+0000","last_fullsized":"2024-09-19T07:31:31.807160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:31.075378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.804966+0000","last_change":"2024-09-19T07:31:12.510999+0000","last_active":"2024-09-19T07:31:31.804966+0000","last_peered":"2024-09-19T07:31:31.804966+0000","last_clean":"2024-09-19T07:31:31.804966+0000","last_became_active":"2024-09-19T07:31:12.510604+0000","last_became_peered":"2024-09-19T07:31:12.510604+0000","last_unstale":"2024-09-19T07:31:31.804966+0000","last_undegraded":"2024-09-19T07:31:31.804966+0000","last_fullsized":"2024-09-19T07:31:31.804966+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:04.779289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805444+0000","last_change":"2024-09-19T07:31:02.428108+0000","last_active":"2024-09-19T07:31:31.805444+0000","last_peered":"2024-09-19T07:31:31.805444+0000","last_clean":"2024-09-19T07:31:31.805444+0000","last_became_active":"2024-09-19T07:31:02.425537+0000","last_became_peered":"2024-09-19T07:31:02.425537+0000","last_unstale":"2024-09-19T07:31:31.805444+0000","last_undegraded":"2024-09-19T07:31:31.805444+0000","last_fullsized":"2024-09-19T07:31:31.805444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:36.094089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:50.896840+0000","last_change":"2024-09-19T07:30:59.404687+0000","last_active":"2024-09-19T07:31:50.896840+0000","last_peered":"2024-09-19T07:31:50.896840+0000","last_clean":"2024-09-19T07:31:50.896840+0000","last_became_active":"2024-09-19T07:30:59.404222+0000","last_became_peered":"2024-09-19T07:30:59.404222+0000","last_unstale":"2024-09-19T07:31:50.896840+0000","last_undegraded":"2024-09-19T07:31:50.896840+0000","last_fullsized":"2024-09-19T07:31:50.896840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:11.287464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:50.895495+0000","last_change":"2024-09-19T07:30:59.414711+0000","last_active":"2024-09-19T07:31:50.895495+0000","last_peered":"2024-09-19T07:31:50.895495+0000","last_clean":"2024-09-19T07:31:50.895495+0000","last_became_active":"2024-09-19T07:30:59.414349+0000","last_became_peered":"2024-09-19T07:30:59.414349+0000","last_unstale":"2024-09-19T07:31:50.895495+0000","last_undegraded":"2024-09-19T07:31:50.895495+0000","last_fullsized":"2024-09-19T07:31:50.895495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:57:09.910328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.810970+0000","last_change":"2024-09-19T07:31:31.810186+0000","last_active":"2024-09-19T07:31:31.810970+0000","last_peered":"2024-09-19T07:31:31.810970+0000","last_clean":"2024-09-19T07:31:31.810970+0000","last_became_active":"2024-09-19T07:31:31.809861+0000","last_became_peered":"2024-09-19T07:31:31.809861+0000","last_unstale":"2024-09-19T07:31:31.810970+0000","last_undegraded":"2024-09-19T07:31:31.810970+0000","last_fullsized":"2024-09-19T07:31:31.810970+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:49.443012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803860+0000","last_change":"2024-09-19T07:31:12.506060+0000","last_active":"2024-09-19T07:31:31.803860+0000","last_peered":"2024-09-19T07:31:31.803860+0000","last_clean":"2024-09-19T07:31:31.803860+0000","last_became_active":"2024-09-19T07:31:12.505535+0000","last_became_peered":"2024-09-19T07:31:12.505535+0000","last_unstale":"2024-09-19T07:31:31.803860+0000","last_undegraded":"2024-09-19T07:31:31.803860+0000","last_fullsized":"2024-09-19T07:31:31.803860+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:02.820475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'6","reported_seq":67,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:32:08.922831+0000","last_change":"2024-09-19T07:30:59.408594+0000","last_active":"2024-09-19T07:32:08.922831+0000","last_peered":"2024-09-19T07:32:08.922831+0000","last_clean":"2024-09-19T07:32:08.922831+0000","last_became_active":"2024-09-19T07:30:59.408164+0000","last_became_peered":"2024-09-19T07:30:59.408164+0000","last_unstale":"2024-09-19T07:32:08.922831+0000","last_undegraded":"2024-09-19T07:32:08.922831+0000","last_fullsized":"2024-09-19T07:32:08.922831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:23.950637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.246515+0000","last_change":"2024-09-19T07:31:02.419940+0000","last_active":"2024-09-19T07:31:32.246515+0000","last_peered":"2024-09-19T07:31:32.246515+0000","last_clean":"2024-09-19T07:31:32.246515+0000","last_became_active":"2024-09-19T07:31:02.419618+0000","last_became_peered":"2024-09-19T07:31:02.419618+0000","last_unstale":"2024-09-19T07:31:32.246515+0000","last_undegraded":"2024-09-19T07:31:32.246515+0000","last_fullsized":"2024-09-19T07:31:32.246515+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:18.208200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803990+0000","last_change":"2024-09-19T07:31:12.497271+0000","last_active":"2024-09-19T07:31:31.803990+0000","last_peered":"2024-09-19T07:31:31.803990+0000","last_clean":"2024-09-19T07:31:31.803990+0000","last_became_active":"2024-09-19T07:31:12.496882+0000","last_became_peered":"2024-09-19T07:31:12.496882+0000","last_unstale":"2024-09-19T07:31:31.803990+0000","last_undegraded":"2024-09-19T07:31:31.803990+0000","last_fullsized":"2024-09-19T07:31:31.803990+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:48.714941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.804672+0000","last_change":"2024-09-19T07:31:02.429917+0000","last_active":"2024-09-19T07:31:31.804672+0000","last_peered":"2024-09-19T07:31:31.804672+0000","last_clean":"2024-09-19T07:31:31.804672+0000","last_became_active":"2024-09-19T07:31:02.429535+0000","last_became_peered":"2024-09-19T07:31:02.429535+0000","last_unstale":"2024-09-19T07:31:31.804672+0000","last_undegraded":"2024-09-19T07:31:31.804672+0000","last_fullsized":"2024-09-19T07:31:31.804672+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:21.958491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803870+0000","last_change":"2024-09-19T07:31:02.433135+0000","last_active":"2024-09-19T07:31:31.803870+0000","last_peered":"2024-09-19T07:31:31.803870+0000","last_clean":"2024-09-19T07:31:31.803870+0000","last_became_active":"2024-09-19T07:31:02.432800+0000","last_became_peered":"2024-09-19T07:31:02.432800+0000","last_unstale":"2024-09-19T07:31:31.803870+0000","last_undegraded":"2024-09-19T07:31:31.803870+0000","last_fullsized":"2024-09-19T07:31:31.803870+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:01.559565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803436+0000","last_change":"2024-09-19T07:31:02.438670+0000","last_active":"2024-09-19T07:31:31.803436+0000","last_peered":"2024-09-19T07:31:31.803436+0000","last_clean":"2024-09-19T07:31:31.803436+0000","last_became_active":"2024-09-19T07:31:02.438191+0000","last_became_peered":"2024-09-19T07:31:02.438191+0000","last_unstale":"2024-09-19T07:31:31.803436+0000","last_undegraded":"2024-09-19T07:31:31.803436+0000","last_fullsized":"2024-09-19T07:31:31.803436+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:55.317996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.019815+0000","last_change":"2024-09-19T07:31:02.434826+0000","last_active":"2024-09-19T07:31:32.019815+0000","last_peered":"2024-09-19T07:31:32.019815+0000","last_clean":"2024-09-19T07:31:32.019815+0000","last_became_active":"2024-09-19T07:31:02.433695+0000","last_became_peered":"2024-09-19T07:31:02.433695+0000","last_unstale":"2024-09-19T07:31:32.019815+0000","last_undegraded":"2024-09-19T07:31:32.019815+0000","last_fullsized":"2024-09-19T07:31:32.019815+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:33:28.866345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.021091+0000","last_change":"2024-09-19T07:31:32.021091+0000","last_active":"2024-09-19T07:31:32.021091+0000","last_peered":"2024-09-19T07:31:32.021091+0000","last_clean":"2024-09-19T07:31:32.021091+0000","last_became_active":"2024-09-19T07:31:32.020537+0000","last_became_peered":"2024-09-19T07:31:32.020537+0000","last_unstale":"2024-09-19T07:31:32.021091+0000","last_undegraded":"2024-09-19T07:31:32.021091+0000","last_fullsized":"2024-09-19T07:31:32.021091+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:31.649790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805242+0000","last_change":"2024-09-19T07:31:02.424514+0000","last_active":"2024-09-19T07:31:31.805242+0000","last_peered":"2024-09-19T07:31:31.805242+0000","last_clean":"2024-09-19T07:31:31.805242+0000","last_became_active":"2024-09-19T07:31:02.424194+0000","last_became_peered":"2024-09-19T07:31:02.424194+0000","last_unstale":"2024-09-19T07:31:31.805242+0000","last_undegraded":"2024-09-19T07:31:31.805242+0000","last_fullsized":"2024-09-19T07:31:31.805242+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:41:47.039372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803999+0000","last_change":"2024-09-19T07:31:02.429624+0000","last_active":"2024-09-19T07:31:31.803999+0000","last_peered":"2024-09-19T07:31:31.803999+0000","last_clean":"2024-09-19T07:31:31.803999+0000","last_became_active":"2024-09-19T07:31:02.429316+0000","last_became_peered":"2024-09-19T07:31:02.429316+0000","last_unstale":"2024-09-19T07:31:31.803999+0000","last_undegraded":"2024-09-19T07:31:31.803999+0000","last_fullsized":"2024-09-19T07:31:31.803999+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:15.736433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.811843+0000","last_change":"2024-09-19T07:31:31.810511+0000","last_active":"2024-09-19T07:31:31.811843+0000","last_peered":"2024-09-19T07:31:31.811843+0000","last_clean":"2024-09-19T07:31:31.811843+0000","last_became_active":"2024-09-19T07:31:31.809805+0000","last_became_peered":"2024-09-19T07:31:31.809805+0000","last_unstale":"2024-09-19T07:31:31.811843+0000","last_undegraded":"2024-09-19T07:31:31.811843+0000","last_fullsized":"2024-09-19T07:31:31.811843+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:48.457693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806855+0000","last_change":"2024-09-19T07:31:02.433859+0000","last_active":"2024-09-19T07:31:31.806855+0000","last_peered":"2024-09-19T07:31:31.806855+0000","last_clean":"2024-09-19T07:31:31.806855+0000","last_became_active":"2024-09-19T07:31:02.433547+0000","last_became_peered":"2024-09-19T07:31:02.433547+0000","last_unstale":"2024-09-19T07:31:31.806855+0000","last_undegraded":"2024-09-19T07:31:31.806855+0000","last_fullsized":"2024-09-19T07:31:31.806855+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:09.623326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805944+0000","last_change":"2024-09-19T07:31:02.442174+0000","last_active":"2024-09-19T07:31:31.805944+0000","last_peered":"2024-09-19T07:31:31.805944+0000","last_clean":"2024-09-19T07:31:31.805944+0000","last_became_active":"2024-09-19T07:31:02.441851+0000","last_became_peered":"2024-09-19T07:31:02.441851+0000","last_unstale":"2024-09-19T07:31:31.805944+0000","last_undegraded":"2024-09-19T07:31:31.805944+0000","last_fullsized":"2024-09-19T07:31:31.805944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:59.235498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.245700+0000","last_change":"2024-09-19T07:31:02.439526+0000","last_active":"2024-09-19T07:31:32.245700+0000","last_peered":"2024-09-19T07:31:32.245700+0000","last_clean":"2024-09-19T07:31:32.245700+0000","last_became_active":"2024-09-19T07:31:02.438996+0000","last_became_peered":"2024-09-19T07:31:02.438996+0000","last_unstale":"2024-09-19T07:31:32.245700+0000","last_undegraded":"2024-09-19T07:31:32.245700+0000","last_fullsized":"2024-09-19T07:31:32.245700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:55.909727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803030+0000","last_change":"2024-09-19T07:31:02.427770+0000","last_active":"2024-09-19T07:31:31.803030+0000","last_peered":"2024-09-19T07:31:31.803030+0000","last_clean":"2024-09-19T07:31:31.803030+0000","last_became_active":"2024-09-19T07:31:02.427208+0000","last_became_peered":"2024-09-19T07:31:02.427208+0000","last_unstale":"2024-09-19T07:31:31.803030+0000","last_undegraded":"2024-09-19T07:31:31.803030+0000","last_fullsized":"2024-09-19T07:31:31.803030+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:01.171781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805785+0000","last_change":"2024-09-19T07:31:02.433089+0000","last_active":"2024-09-19T07:31:31.805785+0000","last_peered":"2024-09-19T07:31:31.805785+0000","last_clean":"2024-09-19T07:31:31.805785+0000","last_became_active":"2024-09-19T07:31:02.432744+0000","last_became_peered":"2024-09-19T07:31:02.432744+0000","last_unstale":"2024-09-19T07:31:31.805785+0000","last_undegraded":"2024-09-19T07:31:31.805785+0000","last_fullsized":"2024-09-19T07:31:31.805785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:32.959555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.804158+0000","last_change":"2024-09-19T07:31:02.435482+0000","last_active":"2024-09-19T07:31:31.804158+0000","last_peered":"2024-09-19T07:31:31.804158+0000","last_clean":"2024-09-19T07:31:31.804158+0000","last_became_active":"2024-09-19T07:31:02.435164+0000","last_became_peered":"2024-09-19T07:31:02.435164+0000","last_unstale":"2024-09-19T07:31:31.804158+0000","last_undegraded":"2024-09-19T07:31:31.804158+0000","last_fullsized":"2024-09-19T07:31:31.804158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:54.478049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.809318+0000","last_change":"2024-09-19T07:31:02.426768+0000","last_active":"2024-09-19T07:31:31.809318+0000","last_peered":"2024-09-19T07:31:31.809318+0000","last_clean":"2024-09-19T07:31:31.809318+0000","last_became_active":"2024-09-19T07:31:02.425592+0000","last_became_peered":"2024-09-19T07:31:02.425592+0000","last_unstale":"2024-09-19T07:31:31.809318+0000","last_undegraded":"2024-09-19T07:31:31.809318+0000","last_fullsized":"2024-09-19T07:31:31.809318+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:44.985532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806121+0000","last_change":"2024-09-19T07:31:02.426233+0000","last_active":"2024-09-19T07:31:31.806121+0000","last_peered":"2024-09-19T07:31:31.806121+0000","last_clean":"2024-09-19T07:31:31.806121+0000","last_became_active":"2024-09-19T07:31:02.425777+0000","last_became_peered":"2024-09-19T07:31:02.425777+0000","last_unstale":"2024-09-19T07:31:31.806121+0000","last_undegraded":"2024-09-19T07:31:31.806121+0000","last_fullsized":"2024-09-19T07:31:31.806121+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:31.497641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.245261+0000","last_change":"2024-09-19T07:31:02.428671+0000","last_active":"2024-09-19T07:31:32.245261+0000","last_peered":"2024-09-19T07:31:32.245261+0000","last_clean":"2024-09-19T07:31:32.245261+0000","last_became_active":"2024-09-19T07:31:02.425930+0000","last_became_peered":"2024-09-19T07:31:02.425930+0000","last_unstale":"2024-09-19T07:31:32.245261+0000","last_undegraded":"2024-09-19T07:31:32.245261+0000","last_fullsized":"2024-09-19T07:31:32.245261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:21.721694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.019802+0000","last_change":"2024-09-19T07:31:02.441450+0000","last_active":"2024-09-19T07:31:32.019802+0000","last_peered":"2024-09-19T07:31:32.019802+0000","last_clean":"2024-09-19T07:31:32.019802+0000","last_became_active":"2024-09-19T07:31:02.436673+0000","last_became_peered":"2024-09-19T07:31:02.436673+0000","last_unstale":"2024-09-19T07:31:32.019802+0000","last_undegraded":"2024-09-19T07:31:32.019802+0000","last_fullsized":"2024-09-19T07:31:32.019802+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:29.744665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.019522+0000","last_change":"2024-09-19T07:31:02.427449+0000","last_active":"2024-09-19T07:31:32.019522+0000","last_peered":"2024-09-19T07:31:32.019522+0000","last_clean":"2024-09-19T07:31:32.019522+0000","last_became_active":"2024-09-19T07:31:02.426601+0000","last_became_peered":"2024-09-19T07:31:02.426601+0000","last_unstale":"2024-09-19T07:31:32.019522+0000","last_undegraded":"2024-09-19T07:31:32.019522+0000","last_fullsized":"2024-09-19T07:31:32.019522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:36.752529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.803008+0000","last_change":"2024-09-19T07:31:02.424816+0000","last_active":"2024-09-19T07:31:31.803008+0000","last_peered":"2024-09-19T07:31:31.803008+0000","last_clean":"2024-09-19T07:31:31.803008+0000","last_became_active":"2024-09-19T07:31:02.423729+0000","last_became_peered":"2024-09-19T07:31:02.423729+0000","last_unstale":"2024-09-19T07:31:31.803008+0000","last_undegraded":"2024-09-19T07:31:31.803008+0000","last_fullsized":"2024-09-19T07:31:31.803008+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:39.546052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806843+0000","last_change":"2024-09-19T07:31:02.428444+0000","last_active":"2024-09-19T07:31:31.806843+0000","last_peered":"2024-09-19T07:31:31.806843+0000","last_clean":"2024-09-19T07:31:31.806843+0000","last_became_active":"2024-09-19T07:31:02.428121+0000","last_became_peered":"2024-09-19T07:31:02.428121+0000","last_unstale":"2024-09-19T07:31:31.806843+0000","last_undegraded":"2024-09-19T07:31:31.806843+0000","last_fullsized":"2024-09-19T07:31:31.806843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:52.649412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531604+0000","last_change":"2024-09-19T07:31:02.423102+0000","last_active":"2024-09-19T07:31:32.531604+0000","last_peered":"2024-09-19T07:31:32.531604+0000","last_clean":"2024-09-19T07:31:32.531604+0000","last_became_active":"2024-09-19T07:31:02.422672+0000","last_became_peered":"2024-09-19T07:31:02.422672+0000","last_unstale":"2024-09-19T07:31:32.531604+0000","last_undegraded":"2024-09-19T07:31:32.531604+0000","last_fullsized":"2024-09-19T07:31:32.531604+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:45.733320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806031+0000","last_change":"2024-09-19T07:31:02.434599+0000","last_active":"2024-09-19T07:31:31.806031+0000","last_peered":"2024-09-19T07:31:31.806031+0000","last_clean":"2024-09-19T07:31:31.806031+0000","last_became_active":"2024-09-19T07:31:02.434263+0000","last_became_peered":"2024-09-19T07:31:02.434263+0000","last_unstale":"2024-09-19T07:31:31.806031+0000","last_undegraded":"2024-09-19T07:31:31.806031+0000","last_fullsized":"2024-09-19T07:31:31.806031+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:16.104483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.246099+0000","last_change":"2024-09-19T07:31:02.429278+0000","last_active":"2024-09-19T07:31:32.246099+0000","last_peered":"2024-09-19T07:31:32.246099+0000","last_clean":"2024-09-19T07:31:32.246099+0000","last_became_active":"2024-09-19T07:31:02.428991+0000","last_became_peered":"2024-09-19T07:31:02.428991+0000","last_unstale":"2024-09-19T07:31:32.246099+0000","last_undegraded":"2024-09-19T07:31:32.246099+0000","last_fullsized":"2024-09-19T07:31:32.246099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:02.421684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805392+0000","last_change":"2024-09-19T07:31:02.436412+0000","last_active":"2024-09-19T07:31:31.805392+0000","last_peered":"2024-09-19T07:31:31.805392+0000","last_clean":"2024-09-19T07:31:31.805392+0000","last_became_active":"2024-09-19T07:31:02.436101+0000","last_became_peered":"2024-09-19T07:31:02.436101+0000","last_unstale":"2024-09-19T07:31:31.805392+0000","last_undegraded":"2024-09-19T07:31:31.805392+0000","last_fullsized":"2024-09-19T07:31:31.805392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:34.827734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.531039+0000","last_change":"2024-09-19T07:31:02.423414+0000","last_active":"2024-09-19T07:31:32.531039+0000","last_peered":"2024-09-19T07:31:32.531039+0000","last_clean":"2024-09-19T07:31:32.531039+0000","last_became_active":"2024-09-19T07:31:02.423100+0000","last_became_peered":"2024-09-19T07:31:02.423100+0000","last_unstale":"2024-09-19T07:31:32.531039+0000","last_undegraded":"2024-09-19T07:31:32.531039+0000","last_fullsized":"2024-09-19T07:31:32.531039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:15.269095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.805431+0000","last_change":"2024-09-19T07:31:02.434953+0000","last_active":"2024-09-19T07:31:31.805431+0000","last_peered":"2024-09-19T07:31:31.805431+0000","last_clean":"2024-09-19T07:31:31.805431+0000","last_became_active":"2024-09-19T07:31:02.434662+0000","last_became_peered":"2024-09-19T07:31:02.434662+0000","last_unstale":"2024-09-19T07:31:31.805431+0000","last_undegraded":"2024-09-19T07:31:31.805431+0000","last_fullsized":"2024-09-19T07:31:31.805431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:17.132108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.246334+0000","last_change":"2024-09-19T07:31:02.427959+0000","last_active":"2024-09-19T07:31:32.246334+0000","last_peered":"2024-09-19T07:31:32.246334+0000","last_clean":"2024-09-19T07:31:32.246334+0000","last_became_active":"2024-09-19T07:31:02.427671+0000","last_became_peered":"2024-09-19T07:31:02.427671+0000","last_unstale":"2024-09-19T07:31:32.246334+0000","last_undegraded":"2024-09-19T07:31:32.246334+0000","last_fullsized":"2024-09-19T07:31:32.246334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:19.608005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806804+0000","last_change":"2024-09-19T07:31:02.433310+0000","last_active":"2024-09-19T07:31:31.806804+0000","last_peered":"2024-09-19T07:31:31.806804+0000","last_clean":"2024-09-19T07:31:31.806804+0000","last_became_active":"2024-09-19T07:31:02.432922+0000","last_became_peered":"2024-09-19T07:31:02.432922+0000","last_unstale":"2024-09-19T07:31:31.806804+0000","last_undegraded":"2024-09-19T07:31:31.806804+0000","last_fullsized":"2024-09-19T07:31:31.806804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:30.043075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.019884+0000","last_change":"2024-09-19T07:31:02.429452+0000","last_active":"2024-09-19T07:31:32.019884+0000","last_peered":"2024-09-19T07:31:32.019884+0000","last_clean":"2024-09-19T07:31:32.019884+0000","last_became_active":"2024-09-19T07:31:02.429142+0000","last_became_peered":"2024-09-19T07:31:02.429142+0000","last_unstale":"2024-09-19T07:31:32.019884+0000","last_undegraded":"2024-09-19T07:31:32.019884+0000","last_fullsized":"2024-09-19T07:31:32.019884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:42.107179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:32.020039+0000","last_change":"2024-09-19T07:31:02.431197+0000","last_active":"2024-09-19T07:31:32.020039+0000","last_peered":"2024-09-19T07:31:32.020039+0000","last_clean":"2024-09-19T07:31:32.020039+0000","last_became_active":"2024-09-19T07:31:02.430812+0000","last_became_peered":"2024-09-19T07:31:02.430812+0000","last_unstale":"2024-09-19T07:31:32.020039+0000","last_undegraded":"2024-09-19T07:31:32.020039+0000","last_fullsized":"2024-09-19T07:31:32.020039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:10.682008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T07:31:31.806102+0000","last_change":"2024-09-19T07:31:02.426753+0000","last_active":"2024-09-19T07:31:31.806102+0000","last_peered":"2024-09-19T07:31:31.806102+0000","last_clean":"2024-09-19T07:31:31.806102+0000","last_became_active":"2024-09-19T07:31:02.425870+0000","last_became_peered":"2024-09-19T07:31:02.425870+0000","last_unstale":"2024-09-19T07:31:31.806102+0000","last_undegraded":"2024-09-19T07:31:31.806102+0000","last_fullsized":"2024-09-19T07:31:31.806102+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:05.410032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61654,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":143,"num_read_kb":137,"num_write":159,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":168642,"data_compressed":5616,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247636,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351003,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1077248,"data_stored":553197,"data_compressed":26426,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579937,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28208,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716944,"statfs":{"total":95995035648,"available":95966150656,"internally_reserved":0,"allocated":1212416,"data_stored":1198116,"data_compressed":64146,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":557529,"data_compressed":26426,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005100,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":553353,"data_compressed":26426,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":1149657,"data_compressed":62274,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":548609,"data_compressed":26426,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":1144852,"data_compressed":62274,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888130,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":553652,"data_compressed":26426,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":548803,"data_compressed":26426,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":599360,"data_compressed":28298,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":597617,"data_compressed":28298,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1097728,"data_stored":549997,"data_compressed":26426,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":49098,"data_compressed":1872,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":50841,"data_compressed":1872,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":59229,"data_compressed":1872,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:32:15.283 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T07:32:15.283 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T07:32:15.284 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T07:32:15.650 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:15 smithi189 ceph-mon[34851]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:32:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:15 smithi189 ceph-mon[34851]: pgmap v350: 129 pgs: 129 active+clean; 637 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-09-19T07:32:16.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:15 smithi159 ceph-mon[30472]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:32:16.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:15 smithi159 ceph-mon[30472]: pgmap v350: 129 pgs: 129 active+clean; 637 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-09-19T07:32:16.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:15 smithi203 ceph-mon[35335]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:32:16.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:15 smithi203 ceph-mon[35335]: pgmap v350: 129 pgs: 129 active+clean; 637 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-09-19T07:32:16.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.277+0000 7f1c08c00640 1 -- 172.21.15.159:0/2754255437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c04078790 msgr2=0x7f1c04079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:16.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.277+0000 7f1c08c00640 1 --2- 172.21.15.159:0/2754255437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c04078790 0x7f1c04079030 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009ca0 tx=0x7f1bfc02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:32:16.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.277+0000 7f1c08c00640 1 -- 172.21.15.159:0/2754255437 shutdown_connections 2024-09-19T07:32:16.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.277+0000 7f1c08c00640 1 --2- 172.21.15.159:0/2754255437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c04078790 0x7f1c04079030 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:16.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.277+0000 7f1c08c00640 1 --2- 172.21.15.159:0/2754255437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c04077df0 0x7f1c04078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:16.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.277+0000 7f1c08c00640 1 --2- 172.21.15.159:0/2754255437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c04076ba0 0x7f1c04076fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:16.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.277+0000 7f1c08c00640 1 -- 172.21.15.159:0/2754255437 >> 172.21.15.159:0/2754255437 conn(0x7f1c040ff3e0 msgr2=0x7f1c04101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:16.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.277+0000 7f1c08c00640 1 -- 172.21.15.159:0/2754255437 shutdown_connections 2024-09-19T07:32:16.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.277+0000 7f1c08c00640 1 -- 172.21.15.159:0/2754255437 wait complete. 2024-09-19T07:32:16.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.278+0000 7f1c08c00640 1 Processor -- start 2024-09-19T07:32:16.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.278+0000 7f1c08c00640 1 -- start start 2024-09-19T07:32:16.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.278+0000 7f1c08c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c04076ba0 0x7f1c0419e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:16.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.278+0000 7f1c08c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c04077df0 0x7f1c0419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:16.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.278+0000 7f1c08c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c04078790 0x7f1c041a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:16.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.278+0000 7f1c08c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04113d60 con 0x7f1c04077df0 2024-09-19T07:32:16.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.278+0000 7f1c08c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04113be0 con 0x7f1c04078790 2024-09-19T07:32:16.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.278+0000 7f1c08c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04113ee0 con 0x7f1c04076ba0 2024-09-19T07:32:16.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.279+0000 7f1bfbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c04077df0 0x7f1c0419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:16.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.279+0000 7f1bfbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c04077df0 0x7f1c0419e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38658/0 (socket says 172.21.15.159:38658) 2024-09-19T07:32:16.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.279+0000 7f1bfbe00640 1 -- 172.21.15.159:0/699551011 learned_addr learned my addr 172.21.15.159:0/699551011 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:16.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.279+0000 7f1c03400640 1 --2- 172.21.15.159:0/699551011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c04076ba0 0x7f1c0419e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:16.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.279+0000 7f1c03e00640 1 --2- 172.21.15.159:0/699551011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c04078790 0x7f1c041a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:16.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.279+0000 7f1bfbe00640 1 -- 172.21.15.159:0/699551011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c04076ba0 msgr2=0x7f1c0419e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:16.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.279+0000 7f1bfbe00640 1 --2- 172.21.15.159:0/699551011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c04076ba0 0x7f1c0419e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:16.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.279+0000 7f1bfbe00640 1 -- 172.21.15.159:0/699551011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c04078790 msgr2=0x7f1c041a5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:16.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.279+0000 7f1bfbe00640 1 --2- 172.21.15.159:0/699551011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c04078790 0x7f1c041a5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:16.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.279+0000 7f1bfbe00640 1 -- 172.21.15.159:0/699551011 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c041a5e70 con 0x7f1c04077df0 2024-09-19T07:32:16.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.279+0000 7f1bfbe00640 1 --2- 172.21.15.159:0/699551011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c04077df0 0x7f1c0419e780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00d950 tx=0x7f1bec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:16.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.279+0000 7f1c00c00640 1 -- 172.21.15.159:0/699551011 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec017030 con 0x7f1c04077df0 2024-09-19T07:32:16.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.280+0000 7f1c00c00640 1 -- 172.21.15.159:0/699551011 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bec004020 con 0x7f1c04077df0 2024-09-19T07:32:16.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.280+0000 7f1c08c00640 1 -- 172.21.15.159:0/699551011 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c041a6160 con 0x7f1c04077df0 2024-09-19T07:32:16.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.280+0000 7f1c00c00640 1 -- 172.21.15.159:0/699551011 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec0047a0 con 0x7f1c04077df0 2024-09-19T07:32:16.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.280+0000 7f1c08c00640 1 -- 172.21.15.159:0/699551011 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c041a66a0 con 0x7f1c04077df0 2024-09-19T07:32:16.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.281+0000 7f1c08c00640 1 -- 172.21.15.159:0/699551011 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c041105d0 con 0x7f1c04077df0 2024-09-19T07:32:16.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.283+0000 7f1c00c00640 1 -- 172.21.15.159:0/699551011 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1bec0041d0 con 0x7f1c04077df0 2024-09-19T07:32:16.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.284+0000 7f1c00c00640 1 --2- 172.21.15.159:0/699551011 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1be80a3300 0x7f1be80a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:16.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.284+0000 7f1c00c00640 1 -- 172.21.15.159:0/699551011 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f1bec0cf840 con 0x7f1c04077df0 2024-09-19T07:32:16.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.284+0000 7f1c03400640 1 --2- 172.21.15.159:0/699551011 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1be80a3300 0x7f1be80a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:16.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.285+0000 7f1c03400640 1 --2- 172.21.15.159:0/699551011 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1be80a3300 0x7f1be80a57c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009bd0 tx=0x7f1bf4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:16.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.288+0000 7f1c00c00640 1 -- 172.21.15.159:0/699551011 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1bec09c040 con 0x7f1c04077df0 2024-09-19T07:32:16.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.528+0000 7f1c08c00640 1 -- 172.21.15.159:0/699551011 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f1c0402da10 con 0x7f1c04077df0 2024-09-19T07:32:16.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.531+0000 7f1c00c00640 1 -- 172.21.15.159:0/699551011 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f1bec09b940 con 0x7f1c04077df0 2024-09-19T07:32:16.532 INFO:teuthology.orchestra.run.smithi159.stdout:[client.0] 2024-09-19T07:32:16.532 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQCA0+tmE5GjHxAAMsO6+Cgo5PWN2Z5Xc0sVZQ== 2024-09-19T07:32:16.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.534+0000 7f1c08c00640 1 -- 172.21.15.159:0/699551011 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1be80a3300 msgr2=0x7f1be80a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:16.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.534+0000 7f1c08c00640 1 --2- 172.21.15.159:0/699551011 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1be80a3300 0x7f1be80a57c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009bd0 tx=0x7f1bf4005f90 comp rx=0 tx=0).stop 2024-09-19T07:32:16.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.534+0000 7f1c08c00640 1 -- 172.21.15.159:0/699551011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c04077df0 msgr2=0x7f1c0419e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:16.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.534+0000 7f1c08c00640 1 --2- 172.21.15.159:0/699551011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c04077df0 0x7f1c0419e780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00d950 tx=0x7f1bec00de10 comp rx=0 tx=0).stop 2024-09-19T07:32:16.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.534+0000 7f1c08c00640 1 -- 172.21.15.159:0/699551011 shutdown_connections 2024-09-19T07:32:16.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.534+0000 7f1c08c00640 1 --2- 172.21.15.159:0/699551011 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1be80a3300 0x7f1be80a57c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:16.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.534+0000 7f1c08c00640 1 --2- 172.21.15.159:0/699551011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c04078790 0x7f1c041a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:16.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.534+0000 7f1c08c00640 1 --2- 172.21.15.159:0/699551011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c04077df0 0x7f1c0419e780 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:16.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.534+0000 7f1c08c00640 1 --2- 172.21.15.159:0/699551011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c04076ba0 0x7f1c0419e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:16.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.534+0000 7f1c08c00640 1 -- 172.21.15.159:0/699551011 >> 172.21.15.159:0/699551011 conn(0x7f1c040ff3e0 msgr2=0x7f1c04100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:16.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.535+0000 7f1c08c00640 1 -- 172.21.15.159:0/699551011 shutdown_connections 2024-09-19T07:32:16.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:16.535+0000 7f1c08c00640 1 -- 172.21.15.159:0/699551011 wait complete. 2024-09-19T07:32:17.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:16 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/699551011' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T07:32:17.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:16 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/699551011' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T07:32:17.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:16 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/699551011' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T07:32:17.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:16 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/699551011' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T07:32:17.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:16 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/699551011' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T07:32:17.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:16 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/699551011' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T07:32:17.414 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:32:17.414 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T07:32:17.414 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T07:32:17.495 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T07:32:17.495 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T07:32:17.495 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T07:32:17.903 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:18.043 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:17 smithi159 ceph-mon[30472]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.9 KiB/s wr, 9 op/s 2024-09-19T07:32:18.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:17 smithi189 ceph-mon[34851]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.9 KiB/s wr, 9 op/s 2024-09-19T07:32:18.187 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:17 smithi203 ceph-mon[35335]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.9 KiB/s wr, 9 op/s 2024-09-19T07:32:18.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.494+0000 7fd41e400640 1 -- 172.21.15.159:0/2198516978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd418109120 msgr2=0x7fd418110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:18.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.494+0000 7fd41e400640 1 --2- 172.21.15.159:0/2198516978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd418109120 0x7fd418110ca0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd410009ca0 tx=0x7fd41002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:32:18.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.494+0000 7fd41e400640 1 -- 172.21.15.159:0/2198516978 shutdown_connections 2024-09-19T07:32:18.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.494+0000 7fd41e400640 1 --2- 172.21.15.159:0/2198516978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd418109120 0x7fd418110ca0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:18.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.494+0000 7fd41e400640 1 --2- 172.21.15.159:0/2198516978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4181049a0 0x7fd4181089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:18.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.494+0000 7fd41e400640 1 --2- 172.21.15.159:0/2198516978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd418103ff0 0x7fd4181043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:18.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.494+0000 7fd41e400640 1 -- 172.21.15.159:0/2198516978 >> 172.21.15.159:0/2198516978 conn(0x7fd4180ffa90 msgr2=0x7fd418101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:18.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.494+0000 7fd41e400640 1 -- 172.21.15.159:0/2198516978 shutdown_connections 2024-09-19T07:32:18.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.494+0000 7fd41e400640 1 -- 172.21.15.159:0/2198516978 wait complete. 2024-09-19T07:32:18.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.495+0000 7fd41e400640 1 Processor -- start 2024-09-19T07:32:18.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.495+0000 7fd41e400640 1 -- start start 2024-09-19T07:32:18.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.496+0000 7fd41e400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd418103ff0 0x7fd41819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:18.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.496+0000 7fd41e400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4181049a0 0x7fd41819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:18.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.496+0000 7fd41e400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd418109120 0x7fd41819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:18.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.496+0000 7fd41e400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd418114150 con 0x7fd4181049a0 2024-09-19T07:32:18.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.496+0000 7fd41e400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd418113fd0 con 0x7fd418109120 2024-09-19T07:32:18.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.496+0000 7fd41e400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181142d0 con 0x7fd418103ff0 2024-09-19T07:32:18.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.496+0000 7fd416a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4181049a0 0x7fd41819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:18.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.496+0000 7fd416a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4181049a0 0x7fd41819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38680/0 (socket says 172.21.15.159:38680) 2024-09-19T07:32:18.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.496+0000 7fd416a00640 1 -- 172.21.15.159:0/2151199094 learned_addr learned my addr 172.21.15.159:0/2151199094 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:18.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.496+0000 7fd417400640 1 --2- 172.21.15.159:0/2151199094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd418103ff0 0x7fd41819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:18.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.496+0000 7fd417e00640 1 --2- 172.21.15.159:0/2151199094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd418109120 0x7fd41819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:18.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.496+0000 7fd416a00640 1 -- 172.21.15.159:0/2151199094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd418103ff0 msgr2=0x7fd41819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:18.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.496+0000 7fd416a00640 1 --2- 172.21.15.159:0/2151199094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd418103ff0 0x7fd41819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:18.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.496+0000 7fd416a00640 1 -- 172.21.15.159:0/2151199094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd418109120 msgr2=0x7fd41819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:18.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.496+0000 7fd416a00640 1 --2- 172.21.15.159:0/2151199094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd418109120 0x7fd41819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:18.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.496+0000 7fd416a00640 1 -- 172.21.15.159:0/2151199094 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41819f560 con 0x7fd4181049a0 2024-09-19T07:32:18.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.496+0000 7fd417e00640 1 --2- 172.21.15.159:0/2151199094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd418109120 0x7fd41819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:32:18.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.497+0000 7fd416a00640 1 --2- 172.21.15.159:0/2151199094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4181049a0 0x7fd41819aaf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd40400e9f0 tx=0x7fd40400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:18.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.497+0000 7fd40fe00640 1 -- 172.21.15.159:0/2151199094 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40400cd80 con 0x7fd4181049a0 2024-09-19T07:32:18.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.497+0000 7fd40fe00640 1 -- 172.21.15.159:0/2151199094 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd404005500 con 0x7fd4181049a0 2024-09-19T07:32:18.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.497+0000 7fd40fe00640 1 -- 172.21.15.159:0/2151199094 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404010690 con 0x7fd4181049a0 2024-09-19T07:32:18.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.497+0000 7fd41e400640 1 -- 172.21.15.159:0/2151199094 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41819f850 con 0x7fd4181049a0 2024-09-19T07:32:18.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.497+0000 7fd41e400640 1 -- 172.21.15.159:0/2151199094 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4181a7130 con 0x7fd4181049a0 2024-09-19T07:32:18.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.498+0000 7fd41e400640 1 -- 172.21.15.159:0/2151199094 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4181043d0 con 0x7fd4181049a0 2024-09-19T07:32:18.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.501+0000 7fd40fe00640 1 -- 172.21.15.159:0/2151199094 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd4040046c0 con 0x7fd4181049a0 2024-09-19T07:32:18.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.502+0000 7fd40fe00640 1 --2- 172.21.15.159:0/2151199094 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd3fc0a3350 0x7fd3fc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:18.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.502+0000 7fd40fe00640 1 -- 172.21.15.159:0/2151199094 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fd4040d0260 con 0x7fd4181049a0 2024-09-19T07:32:18.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.502+0000 7fd417400640 1 --2- 172.21.15.159:0/2151199094 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd3fc0a3350 0x7fd3fc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:18.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.503+0000 7fd417400640 1 --2- 172.21.15.159:0/2151199094 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd3fc0a3350 0x7fd3fc0a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd4080095f0 tx=0x7fd408005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:18.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.506+0000 7fd40fe00640 1 -- 172.21.15.159:0/2151199094 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd40409ca60 con 0x7fd4181049a0 2024-09-19T07:32:18.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.727+0000 7fd41e400640 1 -- 172.21.15.159:0/2151199094 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd4181a7460 con 0x7fd4181049a0 2024-09-19T07:32:18.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.733+0000 7fd40fe00640 1 -- 172.21.15.159:0/2151199094 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264056 (secure 0 0 0) 0x7fd40409c360 con 0x7fd4181049a0 2024-09-19T07:32:18.738 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:32:18.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.737+0000 7fd41e400640 1 -- 172.21.15.159:0/2151199094 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd3fc0a3350 msgr2=0x7fd3fc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:18.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.737+0000 7fd41e400640 1 --2- 172.21.15.159:0/2151199094 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd3fc0a3350 0x7fd3fc0a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd4080095f0 tx=0x7fd408005f90 comp rx=0 tx=0).stop 2024-09-19T07:32:18.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.737+0000 7fd41e400640 1 -- 172.21.15.159:0/2151199094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4181049a0 msgr2=0x7fd41819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:18.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.737+0000 7fd41e400640 1 --2- 172.21.15.159:0/2151199094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4181049a0 0x7fd41819aaf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd40400e9f0 tx=0x7fd40400edf0 comp rx=0 tx=0).stop 2024-09-19T07:32:18.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.737+0000 7fd41e400640 1 -- 172.21.15.159:0/2151199094 shutdown_connections 2024-09-19T07:32:18.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.737+0000 7fd41e400640 1 --2- 172.21.15.159:0/2151199094 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd3fc0a3350 0x7fd3fc0a5810 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:18.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.737+0000 7fd41e400640 1 --2- 172.21.15.159:0/2151199094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd418109120 0x7fd41819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:18.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.737+0000 7fd41e400640 1 --2- 172.21.15.159:0/2151199094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4181049a0 0x7fd41819aaf0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:18.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.737+0000 7fd41e400640 1 --2- 172.21.15.159:0/2151199094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd418103ff0 0x7fd41819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:18.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.737+0000 7fd41e400640 1 -- 172.21.15.159:0/2151199094 >> 172.21.15.159:0/2151199094 conn(0x7fd4180ffa90 msgr2=0x7fd418101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:18.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.738+0000 7fd41e400640 1 -- 172.21.15.159:0/2151199094 shutdown_connections 2024-09-19T07:32:18.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:18.738+0000 7fd41e400640 1 -- 172.21.15.159:0/2151199094 wait complete. 2024-09-19T07:32:19.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:18 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/2151199094' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T07:32:19.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:18 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2151199094' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T07:32:19.203 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:18 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/2151199094' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T07:32:19.504 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":4243905339},{"type":"v1","addr":"172.21.15.159:6801","nonce":4243905339}]},"active_addr":"172.21.15.159:6801/4243905339","active_change":"2024-09-19T07:23:30.597046+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.159:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.159:0","nonce":573283438}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.159:0","nonce":2691587722}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.159:0","nonce":2218622084}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.159:0","nonce":912568592}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.159:0","nonce":386501766}]}]} 2024-09-19T07:32:19.511 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T07:32:19.511 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T07:32:19.511 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:32:19.883 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:20.049 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:19 smithi159 ceph-mon[30472]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-19T07:32:20.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:19 smithi189 ceph-mon[34851]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-19T07:32:20.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:19 smithi203 ceph-mon[35335]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-19T07:32:20.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.478+0000 7fa774a00640 1 -- 172.21.15.159:0/4112734397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7700745c0 msgr2=0x7fa7700749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:20.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.478+0000 7fa774a00640 1 --2- 172.21.15.159:0/4112734397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7700745c0 0x7fa7700749a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa760009ca0 tx=0x7fa76002a950 comp rx=0 tx=0).stop 2024-09-19T07:32:20.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.478+0000 7fa774a00640 1 -- 172.21.15.159:0/4112734397 shutdown_connections 2024-09-19T07:32:20.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.478+0000 7fa774a00640 1 --2- 172.21.15.159:0/4112734397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa77010e260 0x7fa770110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:20.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.478+0000 7fa774a00640 1 --2- 172.21.15.159:0/4112734397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa770074ee0 0x7fa77010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:20.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.478+0000 7fa774a00640 1 --2- 172.21.15.159:0/4112734397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7700745c0 0x7fa7700749a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:20.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.478+0000 7fa774a00640 1 -- 172.21.15.159:0/4112734397 >> 172.21.15.159:0/4112734397 conn(0x7fa7700fd740 msgr2=0x7fa7700ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:20.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.478+0000 7fa774a00640 1 -- 172.21.15.159:0/4112734397 shutdown_connections 2024-09-19T07:32:20.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.478+0000 7fa774a00640 1 -- 172.21.15.159:0/4112734397 wait complete. 2024-09-19T07:32:20.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.479+0000 7fa774a00640 1 Processor -- start 2024-09-19T07:32:20.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.479+0000 7fa774a00640 1 -- start start 2024-09-19T07:32:20.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.480+0000 7fa774a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7700745c0 0x7fa77019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:20.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.480+0000 7fa774a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa770074ee0 0x7fa77019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:20.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.480+0000 7fa774a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa77010e260 0x7fa77019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:20.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.480+0000 7fa774a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa770114000 con 0x7fa7700745c0 2024-09-19T07:32:20.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.480+0000 7fa774a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa770113e80 con 0x7fa770074ee0 2024-09-19T07:32:20.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.480+0000 7fa774a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa770114180 con 0x7fa77010e260 2024-09-19T07:32:20.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.480+0000 7fa76f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7700745c0 0x7fa77019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:20.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.480+0000 7fa76f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7700745c0 0x7fa77019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38696/0 (socket says 172.21.15.159:38696) 2024-09-19T07:32:20.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.480+0000 7fa76f400640 1 -- 172.21.15.159:0/1843708010 learned_addr learned my addr 172.21.15.159:0/1843708010 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:20.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.480+0000 7fa76fe00640 1 --2- 172.21.15.159:0/1843708010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa77010e260 0x7fa77019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:20.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.480+0000 7fa766a00640 1 --2- 172.21.15.159:0/1843708010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa770074ee0 0x7fa77019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:20.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.480+0000 7fa76f400640 1 -- 172.21.15.159:0/1843708010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa77010e260 msgr2=0x7fa77019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:20.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.480+0000 7fa76f400640 1 --2- 172.21.15.159:0/1843708010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa77010e260 0x7fa77019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:20.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.480+0000 7fa76f400640 1 -- 172.21.15.159:0/1843708010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa770074ee0 msgr2=0x7fa77019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:20.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.480+0000 7fa76f400640 1 --2- 172.21.15.159:0/1843708010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa770074ee0 0x7fa77019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:20.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.480+0000 7fa76f400640 1 -- 172.21.15.159:0/1843708010 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77019f450 con 0x7fa7700745c0 2024-09-19T07:32:20.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.481+0000 7fa76f400640 1 --2- 172.21.15.159:0/1843708010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7700745c0 0x7fa77019a4a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa760005e30 tx=0x7fa760003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:20.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.481+0000 7fa76cc00640 1 -- 172.21.15.159:0/1843708010 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760034040 con 0x7fa7700745c0 2024-09-19T07:32:20.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.481+0000 7fa76cc00640 1 -- 172.21.15.159:0/1843708010 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa76002fe20 con 0x7fa7700745c0 2024-09-19T07:32:20.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.481+0000 7fa76cc00640 1 -- 172.21.15.159:0/1843708010 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760045480 con 0x7fa7700745c0 2024-09-19T07:32:20.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.481+0000 7fa774a00640 1 -- 172.21.15.159:0/1843708010 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77019f6e0 con 0x7fa7700745c0 2024-09-19T07:32:20.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.481+0000 7fa774a00640 1 -- 172.21.15.159:0/1843708010 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7701a7020 con 0x7fa7700745c0 2024-09-19T07:32:20.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.482+0000 7fa774a00640 1 -- 172.21.15.159:0/1843708010 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7700749a0 con 0x7fa7700745c0 2024-09-19T07:32:20.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.484+0000 7fa76cc00640 1 -- 172.21.15.159:0/1843708010 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fa760037020 con 0x7fa7700745c0 2024-09-19T07:32:20.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.485+0000 7fa76cc00640 1 --2- 172.21.15.159:0/1843708010 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa7540a3350 0x7fa7540a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:20.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.485+0000 7fa76cc00640 1 -- 172.21.15.159:0/1843708010 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fa7600f0c30 con 0x7fa7700745c0 2024-09-19T07:32:20.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.485+0000 7fa766a00640 1 --2- 172.21.15.159:0/1843708010 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa7540a3350 0x7fa7540a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:20.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.486+0000 7fa766a00640 1 --2- 172.21.15.159:0/1843708010 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa7540a3350 0x7fa7540a5810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa77010f5f0 tx=0x7fa75800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:20.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.489+0000 7fa76cc00640 1 -- 172.21.15.159:0/1843708010 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa7600bd540 con 0x7fa7700745c0 2024-09-19T07:32:20.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.666+0000 7fa774a00640 1 -- 172.21.15.159:0/1843708010 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7701093e0 con 0x7fa7700745c0 2024-09-19T07:32:20.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.668+0000 7fa76cc00640 1 -- 172.21.15.159:0/1843708010 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24765 (secure 0 0 0) 0x7fa7600098b0 con 0x7fa7700745c0 2024-09-19T07:32:20.670 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:32:20.670 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":98,"fsid":"c9d18048-7657-11ef-bceb-c7b262605968","created":"2024-09-19T07:22:32.059836+0000","modified":"2024-09-19T07:31:31.799299+0000","last_up_change":"2024-09-19T07:30:03.328697+0000","last_in_change":"2024-09-19T07:29:38.987579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:25:55.202457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:30:58.022056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:31:01.042290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:31:10.977289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"752f5466-fe04-408a-8566-86625947fd3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6803","nonce":2738900830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6805","nonce":2738900830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6809","nonce":2738900830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6807","nonce":2738900830}]},"public_addr":"172.21.15.159:6803/2738900830","cluster_addr":"172.21.15.159:6805/2738900830","heartbeat_back_addr":"172.21.15.159:6809/2738900830","heartbeat_front_addr":"172.21.15.159:6807/2738900830","state":["exists","up"]},{"osd":1,"uuid":"7513adda-ca82-4df4-aa99-3b050b410424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6801","nonce":265696178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6803","nonce":265696178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6807","nonce":265696178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6805","nonce":265696178}]},"public_addr":"172.21.15.189:6801/265696178","cluster_addr":"172.21.15.189:6803/265696178","heartbeat_back_addr":"172.21.15.189:6807/265696178","heartbeat_front_addr":"172.21.15.189:6805/265696178","state":["exists","up"]},{"osd":2,"uuid":"aa03be79-58d8-4eb7-a051-0b46c502b12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6801","nonce":1826565288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6803","nonce":1826565288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6807","nonce":1826565288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6805","nonce":1826565288}]},"public_addr":"172.21.15.203:6801/1826565288","cluster_addr":"172.21.15.203:6803/1826565288","heartbeat_back_addr":"172.21.15.203:6807/1826565288","heartbeat_front_addr":"172.21.15.203:6805/1826565288","state":["exists","up"]},{"osd":3,"uuid":"fcd734f1-f40f-4566-82ab-c4607039f23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6811","nonce":109457380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6813","nonce":109457380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6817","nonce":109457380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6815","nonce":109457380}]},"public_addr":"172.21.15.159:6811/109457380","cluster_addr":"172.21.15.159:6813/109457380","heartbeat_back_addr":"172.21.15.159:6817/109457380","heartbeat_front_addr":"172.21.15.159:6815/109457380","state":["exists","up"]},{"osd":4,"uuid":"bc344c9e-2267-4058-bbe1-9c270327db8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6809","nonce":1036888180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6811","nonce":1036888180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6815","nonce":1036888180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6813","nonce":1036888180}]},"public_addr":"172.21.15.189:6809/1036888180","cluster_addr":"172.21.15.189:6811/1036888180","heartbeat_back_addr":"172.21.15.189:6815/1036888180","heartbeat_front_addr":"172.21.15.189:6813/1036888180","state":["exists","up"]},{"osd":5,"uuid":"86c48303-929d-42f4-bf48-cbb32649f95f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6809","nonce":4212601512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6811","nonce":4212601512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6815","nonce":4212601512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6813","nonce":4212601512}]},"public_addr":"172.21.15.203:6809/4212601512","cluster_addr":"172.21.15.203:6811/4212601512","heartbeat_back_addr":"172.21.15.203:6815/4212601512","heartbeat_front_addr":"172.21.15.203:6813/4212601512","state":["exists","up"]},{"osd":6,"uuid":"f0176ed1-c3a7-4d75-b213-fe88fab4441a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6819","nonce":3817684578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6821","nonce":3817684578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6825","nonce":3817684578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6823","nonce":3817684578}]},"public_addr":"172.21.15.159:6819/3817684578","cluster_addr":"172.21.15.159:6821/3817684578","heartbeat_back_addr":"172.21.15.159:6825/3817684578","heartbeat_front_addr":"172.21.15.159:6823/3817684578","state":["exists","up"]},{"osd":7,"uuid":"921bd59d-9415-4559-8e9f-035d336afd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6817","nonce":3936826456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6819","nonce":3936826456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6823","nonce":3936826456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6821","nonce":3936826456}]},"public_addr":"172.21.15.189:6817/3936826456","cluster_addr":"172.21.15.189:6819/3936826456","heartbeat_back_addr":"172.21.15.189:6823/3936826456","heartbeat_front_addr":"172.21.15.189:6821/3936826456","state":["exists","up"]},{"osd":8,"uuid":"ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6817","nonce":4062926645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6819","nonce":4062926645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6823","nonce":4062926645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6821","nonce":4062926645}]},"public_addr":"172.21.15.203:6817/4062926645","cluster_addr":"172.21.15.203:6819/4062926645","heartbeat_back_addr":"172.21.15.203:6823/4062926645","heartbeat_front_addr":"172.21.15.203:6821/4062926645","state":["exists","up"]},{"osd":9,"uuid":"df29a10f-1c60-41cb-9c2d-bc9f1cb174bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6827","nonce":4185848747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6829","nonce":4185848747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6833","nonce":4185848747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6831","nonce":4185848747}]},"public_addr":"172.21.15.159:6827/4185848747","cluster_addr":"172.21.15.159:6829/4185848747","heartbeat_back_addr":"172.21.15.159:6833/4185848747","heartbeat_front_addr":"172.21.15.159:6831/4185848747","state":["exists","up"]},{"osd":10,"uuid":"ccc440b4-4682-4998-bb53-95e24423cd1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6825","nonce":3018377605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6827","nonce":3018377605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6831","nonce":3018377605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6829","nonce":3018377605}]},"public_addr":"172.21.15.189:6825/3018377605","cluster_addr":"172.21.15.189:6827/3018377605","heartbeat_back_addr":"172.21.15.189:6831/3018377605","heartbeat_front_addr":"172.21.15.189:6829/3018377605","state":["exists","up"]},{"osd":11,"uuid":"e84afc5f-8051-4341-8fd0-3afb482ab4df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6825","nonce":2807503891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6827","nonce":2807503891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6831","nonce":2807503891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6829","nonce":2807503891}]},"public_addr":"172.21.15.203:6825/2807503891","cluster_addr":"172.21.15.203:6827/2807503891","heartbeat_back_addr":"172.21.15.203:6831/2807503891","heartbeat_front_addr":"172.21.15.203:6829/2807503891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:00.093677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:25.477205+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:51.345020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:21.077603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:48.822640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:13.516029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:44.382228+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:10.810700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:36.817155+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:07.728553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:34.747540+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:01.249332+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:0/2383222735":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1561371050":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6800/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:6800/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/2108730247":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/3393992615":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1366863724":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/1628344860":"2024-09-20T07:23:03.089190+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:32:20.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.671+0000 7fa774a00640 1 -- 172.21.15.159:0/1843708010 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa7540a3350 msgr2=0x7fa7540a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:20.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.671+0000 7fa774a00640 1 --2- 172.21.15.159:0/1843708010 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa7540a3350 0x7fa7540a5810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa77010f5f0 tx=0x7fa75800b410 comp rx=0 tx=0).stop 2024-09-19T07:32:20.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.671+0000 7fa774a00640 1 -- 172.21.15.159:0/1843708010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7700745c0 msgr2=0x7fa77019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:20.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.671+0000 7fa774a00640 1 --2- 172.21.15.159:0/1843708010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7700745c0 0x7fa77019a4a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa760005e30 tx=0x7fa760003f90 comp rx=0 tx=0).stop 2024-09-19T07:32:20.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.671+0000 7fa774a00640 1 -- 172.21.15.159:0/1843708010 shutdown_connections 2024-09-19T07:32:20.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.671+0000 7fa774a00640 1 --2- 172.21.15.159:0/1843708010 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa7540a3350 0x7fa7540a5810 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:20.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.671+0000 7fa774a00640 1 --2- 172.21.15.159:0/1843708010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa77010e260 0x7fa77019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:20.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.671+0000 7fa774a00640 1 --2- 172.21.15.159:0/1843708010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa770074ee0 0x7fa77019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:20.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.671+0000 7fa774a00640 1 --2- 172.21.15.159:0/1843708010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7700745c0 0x7fa77019a4a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:20.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.671+0000 7fa774a00640 1 -- 172.21.15.159:0/1843708010 >> 172.21.15.159:0/1843708010 conn(0x7fa7700fd740 msgr2=0x7fa7700fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:20.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.671+0000 7fa774a00640 1 -- 172.21.15.159:0/1843708010 shutdown_connections 2024-09-19T07:32:20.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:20.671+0000 7fa774a00640 1 -- 172.21.15.159:0/1843708010 wait complete. 2024-09-19T07:32:21.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:20 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1843708010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:32:21.124 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:20 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1843708010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:32:21.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:20 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1843708010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:32:21.574 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T07:32:21.574 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:32:21.946 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:22.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:21 smithi189 ceph-mon[34851]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-19T07:32:22.085 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:21 smithi159 ceph-mon[30472]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-19T07:32:22.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:21 smithi203 ceph-mon[35335]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-19T07:32:22.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.530+0000 7f4bdcc00640 1 -- 172.21.15.159:0/4034377536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bd8104010 msgr2=0x7f4bd81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:22.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.530+0000 7f4bdcc00640 1 --2- 172.21.15.159:0/4034377536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bd8104010 0x7f4bd81043f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009900 tx=0x7f4bc802a410 comp rx=0 tx=0).stop 2024-09-19T07:32:22.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.530+0000 7f4bdcc00640 1 -- 172.21.15.159:0/4034377536 shutdown_connections 2024-09-19T07:32:22.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.530+0000 7f4bdcc00640 1 --2- 172.21.15.159:0/4034377536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bd8109140 0x7f4bd8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:22.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.530+0000 7f4bdcc00640 1 --2- 172.21.15.159:0/4034377536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bd81049c0 0x7f4bd8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:22.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.530+0000 7f4bdcc00640 1 --2- 172.21.15.159:0/4034377536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bd8104010 0x7f4bd81043f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:22.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.530+0000 7f4bdcc00640 1 -- 172.21.15.159:0/4034377536 >> 172.21.15.159:0/4034377536 conn(0x7f4bd80ffa50 msgr2=0x7f4bd8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:22.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.531+0000 7f4bdcc00640 1 -- 172.21.15.159:0/4034377536 shutdown_connections 2024-09-19T07:32:22.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.531+0000 7f4bdcc00640 1 -- 172.21.15.159:0/4034377536 wait complete. 2024-09-19T07:32:22.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.531+0000 7f4bdcc00640 1 Processor -- start 2024-09-19T07:32:22.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.532+0000 7f4bdcc00640 1 -- start start 2024-09-19T07:32:22.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.532+0000 7f4bdcc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bd8104010 0x7f4bd819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:22.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.532+0000 7f4bdcc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bd81049c0 0x7f4bd819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:22.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.532+0000 7f4bdcc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bd8109140 0x7f4bd819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:22.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.532+0000 7f4bdcc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd81141b0 con 0x7f4bd8104010 2024-09-19T07:32:22.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.532+0000 7f4bdcc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd8114030 con 0x7f4bd8109140 2024-09-19T07:32:22.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.532+0000 7f4bdcc00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd8114330 con 0x7f4bd81049c0 2024-09-19T07:32:22.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.532+0000 7f4bd5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bd8104010 0x7f4bd819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:22.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.532+0000 7f4bd5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bd8104010 0x7f4bd819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36178/0 (socket says 172.21.15.159:36178) 2024-09-19T07:32:22.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.532+0000 7f4bd5e00640 1 -- 172.21.15.159:0/1443097560 learned_addr learned my addr 172.21.15.159:0/1443097560 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:22.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.532+0000 7f4bd6800640 1 --2- 172.21.15.159:0/1443097560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bd8109140 0x7f4bd819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:22.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.532+0000 7f4bd5400640 1 --2- 172.21.15.159:0/1443097560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bd81049c0 0x7f4bd819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:22.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.533+0000 7f4bd5e00640 1 -- 172.21.15.159:0/1443097560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bd81049c0 msgr2=0x7f4bd819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:22.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.533+0000 7f4bd5e00640 1 --2- 172.21.15.159:0/1443097560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bd81049c0 0x7f4bd819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:22.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.533+0000 7f4bd5e00640 1 -- 172.21.15.159:0/1443097560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bd8109140 msgr2=0x7f4bd819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:22.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.533+0000 7f4bd5e00640 1 --2- 172.21.15.159:0/1443097560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bd8109140 0x7f4bd819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:22.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.533+0000 7f4bd5e00640 1 -- 172.21.15.159:0/1443097560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd819f5e0 con 0x7f4bd8104010 2024-09-19T07:32:22.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.533+0000 7f4bd5e00640 1 --2- 172.21.15.159:0/1443097560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bd8104010 0x7f4bd819a630 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8004770 tx=0x7f4bc8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:22.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.533+0000 7f4bcea00640 1 -- 172.21.15.159:0/1443097560 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc803f070 con 0x7f4bd8104010 2024-09-19T07:32:22.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.533+0000 7f4bcea00640 1 -- 172.21.15.159:0/1443097560 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bc8031ca0 con 0x7f4bd8104010 2024-09-19T07:32:22.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.533+0000 7f4bdcc00640 1 -- 172.21.15.159:0/1443097560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd819f870 con 0x7f4bd8104010 2024-09-19T07:32:22.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.533+0000 7f4bcea00640 1 -- 172.21.15.159:0/1443097560 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8035030 con 0x7f4bd8104010 2024-09-19T07:32:22.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.534+0000 7f4bdcc00640 1 -- 172.21.15.159:0/1443097560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd81a71b0 con 0x7f4bd8104010 2024-09-19T07:32:22.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.535+0000 7f4bdcc00640 1 -- 172.21.15.159:0/1443097560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd81043f0 con 0x7f4bd8104010 2024-09-19T07:32:22.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.538+0000 7f4bcea00640 1 -- 172.21.15.159:0/1443097560 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4bc8042070 con 0x7f4bd8104010 2024-09-19T07:32:22.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.539+0000 7f4bcea00640 1 --2- 172.21.15.159:0/1443097560 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4bbc0a3350 0x7f4bbc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:22.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.539+0000 7f4bcea00640 1 -- 172.21.15.159:0/1443097560 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f4bc80f03a0 con 0x7f4bd8104010 2024-09-19T07:32:22.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.540+0000 7f4bd5400640 1 --2- 172.21.15.159:0/1443097560 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4bbc0a3350 0x7f4bbc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:22.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.540+0000 7f4bd5400640 1 --2- 172.21.15.159:0/1443097560 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4bbc0a3350 0x7f4bbc0a5810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80ffe70 tx=0x7f4bc4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:22.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.543+0000 7f4bcea00640 1 -- 172.21.15.159:0/1443097560 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4bc80bc610 con 0x7f4bd8104010 2024-09-19T07:32:22.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.723+0000 7f4bdcc00640 1 -- 172.21.15.159:0/1443097560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4bd810c320 con 0x7f4bd8104010 2024-09-19T07:32:22.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.726+0000 7f4bcea00640 1 -- 172.21.15.159:0/1443097560 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24765 (secure 0 0 0) 0x7f4bc80c0910 con 0x7f4bd8104010 2024-09-19T07:32:22.727 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:32:22.727 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":98,"fsid":"c9d18048-7657-11ef-bceb-c7b262605968","created":"2024-09-19T07:22:32.059836+0000","modified":"2024-09-19T07:31:31.799299+0000","last_up_change":"2024-09-19T07:30:03.328697+0000","last_in_change":"2024-09-19T07:29:38.987579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:25:55.202457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:30:58.022056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:31:01.042290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:31:10.977289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"752f5466-fe04-408a-8566-86625947fd3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6803","nonce":2738900830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6805","nonce":2738900830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6809","nonce":2738900830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6807","nonce":2738900830}]},"public_addr":"172.21.15.159:6803/2738900830","cluster_addr":"172.21.15.159:6805/2738900830","heartbeat_back_addr":"172.21.15.159:6809/2738900830","heartbeat_front_addr":"172.21.15.159:6807/2738900830","state":["exists","up"]},{"osd":1,"uuid":"7513adda-ca82-4df4-aa99-3b050b410424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6801","nonce":265696178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6803","nonce":265696178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6807","nonce":265696178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6805","nonce":265696178}]},"public_addr":"172.21.15.189:6801/265696178","cluster_addr":"172.21.15.189:6803/265696178","heartbeat_back_addr":"172.21.15.189:6807/265696178","heartbeat_front_addr":"172.21.15.189:6805/265696178","state":["exists","up"]},{"osd":2,"uuid":"aa03be79-58d8-4eb7-a051-0b46c502b12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6801","nonce":1826565288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6803","nonce":1826565288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6807","nonce":1826565288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6805","nonce":1826565288}]},"public_addr":"172.21.15.203:6801/1826565288","cluster_addr":"172.21.15.203:6803/1826565288","heartbeat_back_addr":"172.21.15.203:6807/1826565288","heartbeat_front_addr":"172.21.15.203:6805/1826565288","state":["exists","up"]},{"osd":3,"uuid":"fcd734f1-f40f-4566-82ab-c4607039f23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6811","nonce":109457380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6813","nonce":109457380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6817","nonce":109457380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6815","nonce":109457380}]},"public_addr":"172.21.15.159:6811/109457380","cluster_addr":"172.21.15.159:6813/109457380","heartbeat_back_addr":"172.21.15.159:6817/109457380","heartbeat_front_addr":"172.21.15.159:6815/109457380","state":["exists","up"]},{"osd":4,"uuid":"bc344c9e-2267-4058-bbe1-9c270327db8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6809","nonce":1036888180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6811","nonce":1036888180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6815","nonce":1036888180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6813","nonce":1036888180}]},"public_addr":"172.21.15.189:6809/1036888180","cluster_addr":"172.21.15.189:6811/1036888180","heartbeat_back_addr":"172.21.15.189:6815/1036888180","heartbeat_front_addr":"172.21.15.189:6813/1036888180","state":["exists","up"]},{"osd":5,"uuid":"86c48303-929d-42f4-bf48-cbb32649f95f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6809","nonce":4212601512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6811","nonce":4212601512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6815","nonce":4212601512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6813","nonce":4212601512}]},"public_addr":"172.21.15.203:6809/4212601512","cluster_addr":"172.21.15.203:6811/4212601512","heartbeat_back_addr":"172.21.15.203:6815/4212601512","heartbeat_front_addr":"172.21.15.203:6813/4212601512","state":["exists","up"]},{"osd":6,"uuid":"f0176ed1-c3a7-4d75-b213-fe88fab4441a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6819","nonce":3817684578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6821","nonce":3817684578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6825","nonce":3817684578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6823","nonce":3817684578}]},"public_addr":"172.21.15.159:6819/3817684578","cluster_addr":"172.21.15.159:6821/3817684578","heartbeat_back_addr":"172.21.15.159:6825/3817684578","heartbeat_front_addr":"172.21.15.159:6823/3817684578","state":["exists","up"]},{"osd":7,"uuid":"921bd59d-9415-4559-8e9f-035d336afd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6817","nonce":3936826456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6819","nonce":3936826456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6823","nonce":3936826456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6821","nonce":3936826456}]},"public_addr":"172.21.15.189:6817/3936826456","cluster_addr":"172.21.15.189:6819/3936826456","heartbeat_back_addr":"172.21.15.189:6823/3936826456","heartbeat_front_addr":"172.21.15.189:6821/3936826456","state":["exists","up"]},{"osd":8,"uuid":"ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6817","nonce":4062926645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6819","nonce":4062926645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6823","nonce":4062926645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6821","nonce":4062926645}]},"public_addr":"172.21.15.203:6817/4062926645","cluster_addr":"172.21.15.203:6819/4062926645","heartbeat_back_addr":"172.21.15.203:6823/4062926645","heartbeat_front_addr":"172.21.15.203:6821/4062926645","state":["exists","up"]},{"osd":9,"uuid":"df29a10f-1c60-41cb-9c2d-bc9f1cb174bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6827","nonce":4185848747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6829","nonce":4185848747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6833","nonce":4185848747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6831","nonce":4185848747}]},"public_addr":"172.21.15.159:6827/4185848747","cluster_addr":"172.21.15.159:6829/4185848747","heartbeat_back_addr":"172.21.15.159:6833/4185848747","heartbeat_front_addr":"172.21.15.159:6831/4185848747","state":["exists","up"]},{"osd":10,"uuid":"ccc440b4-4682-4998-bb53-95e24423cd1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6825","nonce":3018377605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6827","nonce":3018377605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6831","nonce":3018377605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6829","nonce":3018377605}]},"public_addr":"172.21.15.189:6825/3018377605","cluster_addr":"172.21.15.189:6827/3018377605","heartbeat_back_addr":"172.21.15.189:6831/3018377605","heartbeat_front_addr":"172.21.15.189:6829/3018377605","state":["exists","up"]},{"osd":11,"uuid":"e84afc5f-8051-4341-8fd0-3afb482ab4df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6825","nonce":2807503891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6827","nonce":2807503891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6831","nonce":2807503891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6829","nonce":2807503891}]},"public_addr":"172.21.15.203:6825/2807503891","cluster_addr":"172.21.15.203:6827/2807503891","heartbeat_back_addr":"172.21.15.203:6831/2807503891","heartbeat_front_addr":"172.21.15.203:6829/2807503891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:00.093677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:25.477205+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:51.345020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:21.077603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:48.822640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:13.516029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:44.382228+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:10.810700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:36.817155+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:07.728553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:34.747540+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:01.249332+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:0/2383222735":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1561371050":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6800/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:6800/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/2108730247":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/3393992615":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1366863724":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/1628344860":"2024-09-20T07:23:03.089190+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:32:22.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.729+0000 7f4bdcc00640 1 -- 172.21.15.159:0/1443097560 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4bbc0a3350 msgr2=0x7f4bbc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:22.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.729+0000 7f4bdcc00640 1 --2- 172.21.15.159:0/1443097560 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4bbc0a3350 0x7f4bbc0a5810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80ffe70 tx=0x7f4bc4005f90 comp rx=0 tx=0).stop 2024-09-19T07:32:22.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.729+0000 7f4bdcc00640 1 -- 172.21.15.159:0/1443097560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bd8104010 msgr2=0x7f4bd819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:22.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.729+0000 7f4bdcc00640 1 --2- 172.21.15.159:0/1443097560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bd8104010 0x7f4bd819a630 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8004770 tx=0x7f4bc8003c10 comp rx=0 tx=0).stop 2024-09-19T07:32:22.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.729+0000 7f4bdcc00640 1 -- 172.21.15.159:0/1443097560 shutdown_connections 2024-09-19T07:32:22.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.729+0000 7f4bdcc00640 1 --2- 172.21.15.159:0/1443097560 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f4bbc0a3350 0x7f4bbc0a5810 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:22.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.729+0000 7f4bdcc00640 1 --2- 172.21.15.159:0/1443097560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bd8109140 0x7f4bd819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:22.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.729+0000 7f4bdcc00640 1 --2- 172.21.15.159:0/1443097560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bd81049c0 0x7f4bd819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:22.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.729+0000 7f4bdcc00640 1 --2- 172.21.15.159:0/1443097560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bd8104010 0x7f4bd819a630 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:22.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.729+0000 7f4bdcc00640 1 -- 172.21.15.159:0/1443097560 >> 172.21.15.159:0/1443097560 conn(0x7f4bd80ffa50 msgr2=0x7f4bd8100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:22.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.729+0000 7f4bdcc00640 1 -- 172.21.15.159:0/1443097560 shutdown_connections 2024-09-19T07:32:22.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:22.729+0000 7f4bdcc00640 1 -- 172.21.15.159:0/1443097560 wait complete. 2024-09-19T07:32:23.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:22 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1443097560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:32:23.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:22 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1443097560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:32:23.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:22 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1443097560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:32:23.626 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T07:32:23.627 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T07:32:23.627 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T07:32:23.628 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T07:32:23.628 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T07:32:23.628 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T07:32:23.629 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T07:32:23.629 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T07:32:23.630 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-19T07:32:23.630 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-19T07:32:23.630 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-19T07:32:23.631 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-19T07:32:23.846 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:23 smithi159 ceph-mon[30472]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-19T07:32:24.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:23 smithi203 ceph-mon[35335]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-19T07:32:24.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:23 smithi189 ceph-mon[34851]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-19T07:32:24.640 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:24.726 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:25.061 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:25.062 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:25.069 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:25.069 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:25.073 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:25.076 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:25.079 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:25.084 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:26.108 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:25 smithi189 ceph-mon[34851]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-19T07:32:26.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:25 smithi159 ceph-mon[30472]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-19T07:32:26.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:25 smithi203 ceph-mon[35335]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-19T07:32:26.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.371+0000 7f45dd200640 1 -- 172.21.15.159:0/2641408456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45d8101df0 msgr2=0x7f45d81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:26.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.371+0000 7f45dd200640 1 --2- 172.21.15.159:0/2641408456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45d8101df0 0x7f45d81021d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009900 tx=0x7f45c802a3f0 comp rx=0 tx=0).stop 2024-09-19T07:32:26.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.371+0000 7f45dd200640 1 -- 172.21.15.159:0/2641408456 shutdown_connections 2024-09-19T07:32:26.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.371+0000 7f45dd200640 1 --2- 172.21.15.159:0/2641408456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d8106f20 0x7f45d810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:26.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.371+0000 7f45dd200640 1 --2- 172.21.15.159:0/2641408456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d81027a0 0x7f45d81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:26.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.371+0000 7f45dd200640 1 --2- 172.21.15.159:0/2641408456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45d8101df0 0x7f45d81021d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:26.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.371+0000 7f45dd200640 1 -- 172.21.15.159:0/2641408456 >> 172.21.15.159:0/2641408456 conn(0x7f45d80fd8b0 msgr2=0x7f45d80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:26.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.371+0000 7f45dd200640 1 -- 172.21.15.159:0/2641408456 shutdown_connections 2024-09-19T07:32:26.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.371+0000 7f45dd200640 1 -- 172.21.15.159:0/2641408456 wait complete. 2024-09-19T07:32:26.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.372+0000 7f45dd200640 1 Processor -- start 2024-09-19T07:32:26.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.373+0000 7f45dd200640 1 -- start start 2024-09-19T07:32:26.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.373+0000 7f45dd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45d8101df0 0x7f45d819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:26.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.373+0000 7f45dd200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d81027a0 0x7f45d819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:26.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.373+0000 7f45dd200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d8106f20 0x7f45d819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:26.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.373+0000 7f45dd200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8111fd0 con 0x7f45d8101df0 2024-09-19T07:32:26.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.373+0000 7f45dd200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8111e50 con 0x7f45d81027a0 2024-09-19T07:32:26.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.373+0000 7f45dd200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8112150 con 0x7f45d8106f20 2024-09-19T07:32:26.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.373+0000 7f45d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45d8101df0 0x7f45d819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:26.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.373+0000 7f45d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45d8101df0 0x7f45d819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36204/0 (socket says 172.21.15.159:36204) 2024-09-19T07:32:26.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.373+0000 7f45d5e00640 1 -- 172.21.15.159:0/762099198 learned_addr learned my addr 172.21.15.159:0/762099198 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:26.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.373+0000 7f45d6800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d8106f20 0x7f45d819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:26.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.373+0000 7f45d5400640 1 --2- 172.21.15.159:0/762099198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d81027a0 0x7f45d819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:26.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.374+0000 7f45d5e00640 1 -- 172.21.15.159:0/762099198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d8106f20 msgr2=0x7f45d819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:26.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.374+0000 7f45d5e00640 1 --2- 172.21.15.159:0/762099198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d8106f20 0x7f45d819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:26.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.374+0000 7f45d5e00640 1 -- 172.21.15.159:0/762099198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d81027a0 msgr2=0x7f45d819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:26.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.374+0000 7f45d5e00640 1 --2- 172.21.15.159:0/762099198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d81027a0 0x7f45d819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:26.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.374+0000 7f45d5e00640 1 -- 172.21.15.159:0/762099198 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d819f560 con 0x7f45d8101df0 2024-09-19T07:32:26.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.374+0000 7f45d5e00640 1 --2- 172.21.15.159:0/762099198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45d8101df0 0x7f45d819a5b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f45c8004770 tx=0x7f45c8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:26.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.374+0000 7f45cea00640 1 -- 172.21.15.159:0/762099198 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c803f070 con 0x7f45d8101df0 2024-09-19T07:32:26.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.374+0000 7f45cea00640 1 -- 172.21.15.159:0/762099198 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c8031ca0 con 0x7f45d8101df0 2024-09-19T07:32:26.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.374+0000 7f45dd200640 1 -- 172.21.15.159:0/762099198 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d819f7f0 con 0x7f45d8101df0 2024-09-19T07:32:26.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.374+0000 7f45cea00640 1 -- 172.21.15.159:0/762099198 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8035030 con 0x7f45d8101df0 2024-09-19T07:32:26.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.374+0000 7f45dd200640 1 -- 172.21.15.159:0/762099198 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d81a7130 con 0x7f45d8101df0 2024-09-19T07:32:26.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.375+0000 7f45dd200640 1 -- 172.21.15.159:0/762099198 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f45d810e160 con 0x7f45d8101df0 2024-09-19T07:32:26.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.377+0000 7f45cea00640 1 -- 172.21.15.159:0/762099198 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f45c8042070 con 0x7f45d8101df0 2024-09-19T07:32:26.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.379+0000 7f45cea00640 1 --2- 172.21.15.159:0/762099198 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f45bc0a32d0 0x7f45bc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:26.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.379+0000 7f45cea00640 1 -- 172.21.15.159:0/762099198 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f45c80f0730 con 0x7f45d8101df0 2024-09-19T07:32:26.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.379+0000 7f45d5400640 1 --2- 172.21.15.159:0/762099198 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f45bc0a32d0 0x7f45bc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:26.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.379+0000 7f45cea00640 1 --2- 172.21.15.159:0/762099198 >> [v2:172.21.15.203:6800/1826565288,v1:172.21.15.203:6801/1826565288] conn(0x7f45bc0ae4a0 0x7f45bc0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:26.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.379+0000 7f45cea00640 1 -- 172.21.15.159:0/762099198 --> [v2:172.21.15.203:6800/1826565288,v1:172.21.15.203:6801/1826565288] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f45bc0b1010 con 0x7f45bc0ae4a0 2024-09-19T07:32:26.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.379+0000 7f45cea00640 1 -- 172.21.15.159:0/762099198 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f45c80f0ae0 con 0x7f45d8101df0 2024-09-19T07:32:26.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.379+0000 7f45d6800640 1 --2- 172.21.15.159:0/762099198 >> [v2:172.21.15.203:6800/1826565288,v1:172.21.15.203:6801/1826565288] conn(0x7f45bc0ae4a0 0x7f45bc0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:26.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.380+0000 7f45d5400640 1 --2- 172.21.15.159:0/762099198 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f45bc0a32d0 0x7f45bc0a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f45d8076480 tx=0x7f45c4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:26.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.380+0000 7f45d6800640 1 --2- 172.21.15.159:0/762099198 >> [v2:172.21.15.203:6800/1826565288,v1:172.21.15.203:6801/1826565288] conn(0x7f45bc0ae4a0 0x7f45bc0b0940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:26.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.381+0000 7f45cea00640 1 -- 172.21.15.159:0/762099198 <== osd.2 v2:172.21.15.203:6800/1826565288 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45bc0b1010 con 0x7f45bc0ae4a0 2024-09-19T07:32:26.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.395+0000 7f45dd200640 1 -- 172.21.15.159:0/762099198 --> [v2:172.21.15.203:6800/1826565288,v1:172.21.15.203:6801/1826565288] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f45d81032d0 con 0x7f45bc0ae4a0 2024-09-19T07:32:26.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.396+0000 7f45cea00640 1 -- 172.21.15.159:0/762099198 <== osd.2 v2:172.21.15.203:6800/1826565288 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f45d81032d0 con 0x7f45bc0ae4a0 2024-09-19T07:32:26.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.396+0000 7f45dd200640 1 -- 172.21.15.159:0/762099198 >> [v2:172.21.15.203:6800/1826565288,v1:172.21.15.203:6801/1826565288] conn(0x7f45bc0ae4a0 msgr2=0x7f45bc0b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:26.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.396+0000 7f45dd200640 1 --2- 172.21.15.159:0/762099198 >> [v2:172.21.15.203:6800/1826565288,v1:172.21.15.203:6801/1826565288] conn(0x7f45bc0ae4a0 0x7f45bc0b0940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:26.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.396+0000 7f45dd200640 1 -- 172.21.15.159:0/762099198 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f45bc0a32d0 msgr2=0x7f45bc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:26.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.396+0000 7f45dd200640 1 --2- 172.21.15.159:0/762099198 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f45bc0a32d0 0x7f45bc0a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f45d8076480 tx=0x7f45c4005f90 comp rx=0 tx=0).stop 2024-09-19T07:32:26.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.396+0000 7f45dd200640 1 -- 172.21.15.159:0/762099198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45d8101df0 msgr2=0x7f45d819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:26.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.396+0000 7f45dd200640 1 --2- 172.21.15.159:0/762099198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45d8101df0 0x7f45d819a5b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f45c8004770 tx=0x7f45c8003c10 comp rx=0 tx=0).stop 2024-09-19T07:32:26.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.397+0000 7f45d6800640 1 -- 172.21.15.159:0/762099198 reap_dead start 2024-09-19T07:32:26.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.397+0000 7f45dd200640 1 -- 172.21.15.159:0/762099198 shutdown_connections 2024-09-19T07:32:26.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.397+0000 7f45dd200640 1 -- 172.21.15.159:0/762099198 >> 172.21.15.159:0/762099198 conn(0x7f45d80fd8b0 msgr2=0x7f45d80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:26.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.397+0000 7f45dd200640 1 -- 172.21.15.159:0/762099198 shutdown_connections 2024-09-19T07:32:26.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.397+0000 7f45dd200640 1 -- 172.21.15.159:0/762099198 wait complete. 2024-09-19T07:32:26.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.655+0000 7f965f200640 1 -- 172.21.15.159:0/1074834765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96601019c0 msgr2=0x7f9660101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:26.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.655+0000 7f965f200640 1 --2- 172.21.15.159:0/1074834765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96601019c0 0x7f9660101e20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f964c009930 tx=0x7f964c02a420 comp rx=0 tx=0).stop 2024-09-19T07:32:26.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.655+0000 7f965f200640 1 -- 172.21.15.159:0/1074834765 shutdown_connections 2024-09-19T07:32:26.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.655+0000 7f965f200640 1 --2- 172.21.15.159:0/1074834765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9660102360 0x7f966010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:26.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.655+0000 7f965f200640 1 --2- 172.21.15.159:0/1074834765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96601019c0 0x7f9660101e20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:26.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.655+0000 7f965f200640 1 --2- 172.21.15.159:0/1074834765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96601079c0 0x7f9660107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:26.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.655+0000 7f965f200640 1 -- 172.21.15.159:0/1074834765 >> 172.21.15.159:0/1074834765 conn(0x7f96600fd740 msgr2=0x7f96600ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:26.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.655+0000 7f965f200640 1 -- 172.21.15.159:0/1074834765 shutdown_connections 2024-09-19T07:32:26.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.655+0000 7f965f200640 1 -- 172.21.15.159:0/1074834765 wait complete. 2024-09-19T07:32:26.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.656+0000 7f965f200640 1 Processor -- start 2024-09-19T07:32:26.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.656+0000 7f965f200640 1 -- start start 2024-09-19T07:32:26.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.657+0000 7f965f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96601019c0 0x7f96601982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:26.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.657+0000 7f965f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9660102360 0x7f9660198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:26.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.657+0000 7f965f200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96601079c0 0x7f966019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:26.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.657+0000 7f965f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9660111dc0 con 0x7f96601019c0 2024-09-19T07:32:26.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.657+0000 7f965f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9660111c40 con 0x7f9660102360 2024-09-19T07:32:26.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.657+0000 7f965f200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9660111f40 con 0x7f96601079c0 2024-09-19T07:32:26.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.657+0000 7f965de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96601019c0 0x7f96601982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:26.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.657+0000 7f965de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96601019c0 0x7f96601982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36222/0 (socket says 172.21.15.159:36222) 2024-09-19T07:32:26.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.657+0000 7f965de00640 1 -- 172.21.15.159:0/2634972517 learned_addr learned my addr 172.21.15.159:0/2634972517 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:26.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.657+0000 7f965d400640 1 --2- 172.21.15.159:0/2634972517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9660102360 0x7f9660198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:26.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.657+0000 7f965e800640 1 --2- 172.21.15.159:0/2634972517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96601079c0 0x7f966019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:26.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.657+0000 7f965de00640 1 -- 172.21.15.159:0/2634972517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96601079c0 msgr2=0x7f966019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:26.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.657+0000 7f965de00640 1 --2- 172.21.15.159:0/2634972517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96601079c0 0x7f966019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:26.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.657+0000 7f965de00640 1 -- 172.21.15.159:0/2634972517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9660102360 msgr2=0x7f9660198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:26.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.657+0000 7f965de00640 1 --2- 172.21.15.159:0/2634972517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9660102360 0x7f9660198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:26.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.658+0000 7f965de00640 1 -- 172.21.15.159:0/2634972517 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966019d2a0 con 0x7f96601019c0 2024-09-19T07:32:26.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.658+0000 7f965d400640 1 --2- 172.21.15.159:0/2634972517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9660102360 0x7f9660198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:32:26.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.658+0000 7f965de00640 1 --2- 172.21.15.159:0/2634972517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96601019c0 0x7f96601982f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f965000e9f0 tx=0x7f965000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:26.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.658+0000 7f9656a00640 1 -- 172.21.15.159:0/2634972517 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965000ccc0 con 0x7f96601019c0 2024-09-19T07:32:26.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.658+0000 7f9656a00640 1 -- 172.21.15.159:0/2634972517 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f965000ce60 con 0x7f96601019c0 2024-09-19T07:32:26.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.658+0000 7f9656a00640 1 -- 172.21.15.159:0/2634972517 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650010690 con 0x7f96601019c0 2024-09-19T07:32:26.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.658+0000 7f965f200640 1 -- 172.21.15.159:0/2634972517 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966019d590 con 0x7f96601019c0 2024-09-19T07:32:26.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.658+0000 7f965f200640 1 -- 172.21.15.159:0/2634972517 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96601a4e70 con 0x7f96601019c0 2024-09-19T07:32:26.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.659+0000 7f965f200640 1 -- 172.21.15.159:0/2634972517 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9660062940 con 0x7f96601019c0 2024-09-19T07:32:26.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.662+0000 7f9656a00640 1 -- 172.21.15.159:0/2634972517 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f96500046c0 con 0x7f96601019c0 2024-09-19T07:32:26.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.663+0000 7f9656a00640 1 --2- 172.21.15.159:0/2634972517 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f96440a3350 0x7f96440a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:26.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.663+0000 7f9656a00640 1 -- 172.21.15.159:0/2634972517 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f96500d05a0 con 0x7f96601019c0 2024-09-19T07:32:26.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.663+0000 7f965d400640 1 --2- 172.21.15.159:0/2634972517 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f96440a3350 0x7f96440a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:26.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.664+0000 7f9656a00640 1 --2- 172.21.15.159:0/2634972517 >> [v2:172.21.15.203:6824/2807503891,v1:172.21.15.203:6825/2807503891] conn(0x7f96440ae520 0x7f96440b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:26.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.664+0000 7f9656a00640 1 -- 172.21.15.159:0/2634972517 --> [v2:172.21.15.203:6824/2807503891,v1:172.21.15.203:6825/2807503891] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f965001aec0 con 0x7f96440ae520 2024-09-19T07:32:26.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.664+0000 7f9656a00640 1 -- 172.21.15.159:0/2634972517 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f96500d0950 con 0x7f96601019c0 2024-09-19T07:32:26.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.664+0000 7f965e800640 1 --2- 172.21.15.159:0/2634972517 >> [v2:172.21.15.203:6824/2807503891,v1:172.21.15.203:6825/2807503891] conn(0x7f96440ae520 0x7f96440b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:26.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.664+0000 7f965d400640 1 --2- 172.21.15.159:0/2634972517 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f96440a3350 0x7f96440a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f964c009900 tx=0x7f964c0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:26.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.665+0000 7f965e800640 1 --2- 172.21.15.159:0/2634972517 >> [v2:172.21.15.203:6824/2807503891,v1:172.21.15.203:6825/2807503891] conn(0x7f96440ae520 0x7f96440b09c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:26.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.665+0000 7f9656a00640 1 -- 172.21.15.159:0/2634972517 <== osd.11 v2:172.21.15.203:6824/2807503891 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f965001aec0 con 0x7f96440ae520 2024-09-19T07:32:26.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.679+0000 7f965f200640 1 -- 172.21.15.159:0/2634972517 --> [v2:172.21.15.203:6824/2807503891,v1:172.21.15.203:6825/2807503891] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9660062dc0 con 0x7f96440ae520 2024-09-19T07:32:26.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.680+0000 7f9656a00640 1 -- 172.21.15.159:0/2634972517 <== osd.11 v2:172.21.15.203:6824/2807503891 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9660062dc0 con 0x7f96440ae520 2024-09-19T07:32:26.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.680+0000 7f965f200640 1 -- 172.21.15.159:0/2634972517 >> [v2:172.21.15.203:6824/2807503891,v1:172.21.15.203:6825/2807503891] conn(0x7f96440ae520 msgr2=0x7f96440b09c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:26.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.680+0000 7f965f200640 1 --2- 172.21.15.159:0/2634972517 >> [v2:172.21.15.203:6824/2807503891,v1:172.21.15.203:6825/2807503891] conn(0x7f96440ae520 0x7f96440b09c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:26.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.680+0000 7f965f200640 1 -- 172.21.15.159:0/2634972517 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f96440a3350 msgr2=0x7f96440a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:26.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.680+0000 7f965f200640 1 --2- 172.21.15.159:0/2634972517 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f96440a3350 0x7f96440a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f964c009900 tx=0x7f964c0058d0 comp rx=0 tx=0).stop 2024-09-19T07:32:26.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.680+0000 7f965f200640 1 -- 172.21.15.159:0/2634972517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96601019c0 msgr2=0x7f96601982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:26.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.680+0000 7f965f200640 1 --2- 172.21.15.159:0/2634972517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96601019c0 0x7f96601982f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f965000e9f0 tx=0x7f965000edf0 comp rx=0 tx=0).stop 2024-09-19T07:32:26.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.680+0000 7f965e800640 1 -- 172.21.15.159:0/2634972517 reap_dead start 2024-09-19T07:32:26.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.681+0000 7f965f200640 1 -- 172.21.15.159:0/2634972517 shutdown_connections 2024-09-19T07:32:26.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.681+0000 7f965f200640 1 -- 172.21.15.159:0/2634972517 >> 172.21.15.159:0/2634972517 conn(0x7f96600fd740 msgr2=0x7f96600fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:26.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.681+0000 7f965f200640 1 -- 172.21.15.159:0/2634972517 shutdown_connections 2024-09-19T07:32:26.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:26.681+0000 7f965f200640 1 -- 172.21.15.159:0/2634972517 wait complete. 2024-09-19T07:32:27.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.010+0000 7ff799800640 1 -- 172.21.15.159:0/224461688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff794103e30 msgr2=0x7ff794104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:27.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.010+0000 7ff799800640 1 --2- 172.21.15.159:0/224461688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff794103e30 0x7ff794104210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff784009900 tx=0x7ff78402a3f0 comp rx=0 tx=0).stop 2024-09-19T07:32:27.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.010+0000 7ff799800640 1 -- 172.21.15.159:0/224461688 shutdown_connections 2024-09-19T07:32:27.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.011+0000 7ff799800640 1 --2- 172.21.15.159:0/224461688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff794108f60 0x7ff794110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:27.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.011+0000 7ff799800640 1 --2- 172.21.15.159:0/224461688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7941047e0 0x7ff794108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:27.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.011+0000 7ff799800640 1 --2- 172.21.15.159:0/224461688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff794103e30 0x7ff794104210 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:27.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.011+0000 7ff799800640 1 -- 172.21.15.159:0/224461688 >> 172.21.15.159:0/224461688 conn(0x7ff7940ff910 msgr2=0x7ff794101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:27.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.011+0000 7ff799800640 1 -- 172.21.15.159:0/224461688 shutdown_connections 2024-09-19T07:32:27.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.012+0000 7ff799800640 1 -- 172.21.15.159:0/224461688 wait complete. 2024-09-19T07:32:27.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.012+0000 7ff799800640 1 Processor -- start 2024-09-19T07:32:27.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.013+0000 7ff799800640 1 -- start start 2024-09-19T07:32:27.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.013+0000 7ff799800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff794103e30 0x7ff79419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:27.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.013+0000 7ff799800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7941047e0 0x7ff79419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:27.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.013+0000 7ff799800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff794108f60 0x7ff79419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:27.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.013+0000 7ff799800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff794114010 con 0x7ff7941047e0 2024-09-19T07:32:27.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.013+0000 7ff799800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff794113e90 con 0x7ff794103e30 2024-09-19T07:32:27.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.013+0000 7ff799800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff794114190 con 0x7ff794108f60 2024-09-19T07:32:27.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.013+0000 7ff793e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff794103e30 0x7ff79419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:27.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.013+0000 7ff793400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7941047e0 0x7ff79419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:27.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.013+0000 7ff793400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7941047e0 0x7ff79419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36232/0 (socket says 172.21.15.159:36232) 2024-09-19T07:32:27.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.013+0000 7ff798e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff794108f60 0x7ff79419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:27.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.013+0000 7ff793400640 1 -- 172.21.15.159:0/652092887 learned_addr learned my addr 172.21.15.159:0/652092887 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:27.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.013+0000 7ff793e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff794103e30 0x7ff79419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:52006/0 (socket says 172.21.15.159:52006) 2024-09-19T07:32:27.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.014+0000 7ff793e00640 1 -- 172.21.15.159:0/652092887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff794108f60 msgr2=0x7ff79419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:27.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.014+0000 7ff793e00640 1 --2- 172.21.15.159:0/652092887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff794108f60 0x7ff79419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:27.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.014+0000 7ff793e00640 1 -- 172.21.15.159:0/652092887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7941047e0 msgr2=0x7ff79419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:27.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.014+0000 7ff793e00640 1 --2- 172.21.15.159:0/652092887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7941047e0 0x7ff79419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:27.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.014+0000 7ff793e00640 1 -- 172.21.15.159:0/652092887 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79419f440 con 0x7ff794103e30 2024-09-19T07:32:27.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.014+0000 7ff793e00640 1 --2- 172.21.15.159:0/652092887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff794103e30 0x7ff79419a490 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff78402a910 tx=0x7ff784003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:27.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.014+0000 7ff790c00640 1 -- 172.21.15.159:0/652092887 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78403f070 con 0x7ff794103e30 2024-09-19T07:32:27.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.014+0000 7ff790c00640 1 -- 172.21.15.159:0/652092887 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff784031c90 con 0x7ff794103e30 2024-09-19T07:32:27.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.014+0000 7ff799800640 1 -- 172.21.15.159:0/652092887 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79419f6d0 con 0x7ff794103e30 2024-09-19T07:32:27.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.015+0000 7ff790c00640 1 -- 172.21.15.159:0/652092887 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784035030 con 0x7ff794103e30 2024-09-19T07:32:27.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.015+0000 7ff799800640 1 -- 172.21.15.159:0/652092887 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7941a7010 con 0x7ff794103e30 2024-09-19T07:32:27.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.016+0000 7ff799800640 1 -- 172.21.15.159:0/652092887 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff794062980 con 0x7ff794103e30 2024-09-19T07:32:27.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.018+0000 7ff790c00640 1 -- 172.21.15.159:0/652092887 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff784042070 con 0x7ff794103e30 2024-09-19T07:32:27.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.020+0000 7ff790c00640 1 --2- 172.21.15.159:0/652092887 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff7780a32d0 0x7ff7780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:27.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.020+0000 7ff790c00640 1 -- 172.21.15.159:0/652092887 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7ff7840f0cf0 con 0x7ff794103e30 2024-09-19T07:32:27.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.020+0000 7ff793400640 1 --2- 172.21.15.159:0/652092887 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff7780a32d0 0x7ff7780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:27.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.020+0000 7ff790c00640 1 --2- 172.21.15.159:0/652092887 >> [v2:172.21.15.189:6824/3018377605,v1:172.21.15.189:6825/3018377605] conn(0x7ff7780ae4a0 0x7ff7780b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:27.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.020+0000 7ff790c00640 1 -- 172.21.15.159:0/652092887 --> [v2:172.21.15.189:6824/3018377605,v1:172.21.15.189:6825/3018377605] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff784038e90 con 0x7ff7780ae4a0 2024-09-19T07:32:27.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.020+0000 7ff790c00640 1 -- 172.21.15.159:0/652092887 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7ff7840f10a0 con 0x7ff794103e30 2024-09-19T07:32:27.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.021+0000 7ff793400640 1 --2- 172.21.15.159:0/652092887 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff7780a32d0 0x7ff7780a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff7940ffd30 tx=0x7ff780005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:27.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.021+0000 7ff798e00640 1 --2- 172.21.15.159:0/652092887 >> [v2:172.21.15.189:6824/3018377605,v1:172.21.15.189:6825/3018377605] conn(0x7ff7780ae4a0 0x7ff7780b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:27.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.021+0000 7ff798e00640 1 --2- 172.21.15.159:0/652092887 >> [v2:172.21.15.189:6824/3018377605,v1:172.21.15.189:6825/3018377605] conn(0x7ff7780ae4a0 0x7ff7780b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:27.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.022+0000 7ff790c00640 1 -- 172.21.15.159:0/652092887 <== osd.10 v2:172.21.15.189:6824/3018377605 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff784038e90 con 0x7ff7780ae4a0 2024-09-19T07:32:27.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.041+0000 7ff799800640 1 -- 172.21.15.159:0/652092887 --> [v2:172.21.15.189:6824/3018377605,v1:172.21.15.189:6825/3018377605] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff794062e00 con 0x7ff7780ae4a0 2024-09-19T07:32:27.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.042+0000 7ff790c00640 1 -- 172.21.15.159:0/652092887 <== osd.10 v2:172.21.15.189:6824/3018377605 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff794062e00 con 0x7ff7780ae4a0 2024-09-19T07:32:27.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.042+0000 7ff799800640 1 -- 172.21.15.159:0/652092887 >> [v2:172.21.15.189:6824/3018377605,v1:172.21.15.189:6825/3018377605] conn(0x7ff7780ae4a0 msgr2=0x7ff7780b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:27.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.042+0000 7ff799800640 1 --2- 172.21.15.159:0/652092887 >> [v2:172.21.15.189:6824/3018377605,v1:172.21.15.189:6825/3018377605] conn(0x7ff7780ae4a0 0x7ff7780b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:27.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.042+0000 7ff799800640 1 -- 172.21.15.159:0/652092887 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff7780a32d0 msgr2=0x7ff7780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:27.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.042+0000 7ff799800640 1 --2- 172.21.15.159:0/652092887 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff7780a32d0 0x7ff7780a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff7940ffd30 tx=0x7ff780005f90 comp rx=0 tx=0).stop 2024-09-19T07:32:27.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.042+0000 7ff799800640 1 -- 172.21.15.159:0/652092887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff794103e30 msgr2=0x7ff79419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:27.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.042+0000 7ff799800640 1 --2- 172.21.15.159:0/652092887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff794103e30 0x7ff79419a490 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff78402a910 tx=0x7ff784003b20 comp rx=0 tx=0).stop 2024-09-19T07:32:27.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.042+0000 7ff798e00640 1 -- 172.21.15.159:0/652092887 reap_dead start 2024-09-19T07:32:27.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.042+0000 7ff799800640 1 -- 172.21.15.159:0/652092887 shutdown_connections 2024-09-19T07:32:27.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.042+0000 7ff799800640 1 -- 172.21.15.159:0/652092887 >> 172.21.15.159:0/652092887 conn(0x7ff7940ff910 msgr2=0x7ff794100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:27.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.042+0000 7ff799800640 1 -- 172.21.15.159:0/652092887 shutdown_connections 2024-09-19T07:32:27.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.043+0000 7ff799800640 1 -- 172.21.15.159:0/652092887 wait complete. 2024-09-19T07:32:27.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.669+0000 7fd3d5400640 1 -- 172.21.15.159:0/775404527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3d0101de0 msgr2=0x7fd3d01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:27.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.669+0000 7fd3d5400640 1 --2- 172.21.15.159:0/775404527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3d0101de0 0x7fd3d01021c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009860 tx=0x7fd3c002a340 comp rx=0 tx=0).stop 2024-09-19T07:32:27.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.669+0000 7fd3d5400640 1 -- 172.21.15.159:0/775404527 shutdown_connections 2024-09-19T07:32:27.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.669+0000 7fd3d5400640 1 --2- 172.21.15.159:0/775404527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d0106f10 0x7fd3d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:27.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.669+0000 7fd3d5400640 1 --2- 172.21.15.159:0/775404527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3d0102790 0x7fd3d01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:27.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.669+0000 7fd3d5400640 1 --2- 172.21.15.159:0/775404527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3d0101de0 0x7fd3d01021c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:27.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.669+0000 7fd3d5400640 1 -- 172.21.15.159:0/775404527 >> 172.21.15.159:0/775404527 conn(0x7fd3d00fd8c0 msgr2=0x7fd3d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:27.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.669+0000 7fd3d5400640 1 -- 172.21.15.159:0/775404527 shutdown_connections 2024-09-19T07:32:27.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.670+0000 7fd3d5400640 1 -- 172.21.15.159:0/775404527 wait complete. 2024-09-19T07:32:27.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.671+0000 7fd3d5400640 1 Processor -- start 2024-09-19T07:32:27.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.671+0000 7fd3d5400640 1 -- start start 2024-09-19T07:32:27.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.671+0000 7fd3d5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d0101de0 0x7fd3d019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:27.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.671+0000 7fd3d5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3d0102790 0x7fd3d019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:27.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.671+0000 7fd3d5400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3d0106f10 0x7fd3d019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:27.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.671+0000 7fd3d5400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d0111f70 con 0x7fd3d0101de0 2024-09-19T07:32:27.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.671+0000 7fd3d5400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d0111df0 con 0x7fd3d0102790 2024-09-19T07:32:27.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.671+0000 7fd3d5400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01120f0 con 0x7fd3d0106f10 2024-09-19T07:32:27.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.671+0000 7fd3ce800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d0101de0 0x7fd3d019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:27.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.672+0000 7fd3ce800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d0101de0 0x7fd3d019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36264/0 (socket says 172.21.15.159:36264) 2024-09-19T07:32:27.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.672+0000 7fd3ce800640 1 -- 172.21.15.159:0/4078649386 learned_addr learned my addr 172.21.15.159:0/4078649386 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:27.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.672+0000 7fd3cf200640 1 --2- 172.21.15.159:0/4078649386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3d0106f10 0x7fd3d019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:27.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.672+0000 7fd3cde00640 1 --2- 172.21.15.159:0/4078649386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3d0102790 0x7fd3d019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:27.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.672+0000 7fd3ce800640 1 -- 172.21.15.159:0/4078649386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3d0106f10 msgr2=0x7fd3d019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:27.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.672+0000 7fd3ce800640 1 --2- 172.21.15.159:0/4078649386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3d0106f10 0x7fd3d019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:27.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.672+0000 7fd3ce800640 1 -- 172.21.15.159:0/4078649386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3d0102790 msgr2=0x7fd3d019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:27.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.672+0000 7fd3ce800640 1 --2- 172.21.15.159:0/4078649386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3d0102790 0x7fd3d019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:27.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.672+0000 7fd3ce800640 1 -- 172.21.15.159:0/4078649386 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d019f5f0 con 0x7fd3d0101de0 2024-09-19T07:32:27.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.672+0000 7fd3ce800640 1 --2- 172.21.15.159:0/4078649386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d0101de0 0x7fd3d019a640 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0035040 tx=0x7fd3c0033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:27.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.673+0000 7fd3c7400640 1 -- 172.21.15.159:0/4078649386 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0037040 con 0x7fd3d0101de0 2024-09-19T07:32:27.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.673+0000 7fd3c7400640 1 -- 172.21.15.159:0/4078649386 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3c0040070 con 0x7fd3d0101de0 2024-09-19T07:32:27.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.673+0000 7fd3d5400640 1 -- 172.21.15.159:0/4078649386 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d019f880 con 0x7fd3d0101de0 2024-09-19T07:32:27.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.673+0000 7fd3c7400640 1 -- 172.21.15.159:0/4078649386 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c00039b0 con 0x7fd3d0101de0 2024-09-19T07:32:27.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.673+0000 7fd3d5400640 1 -- 172.21.15.159:0/4078649386 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d01a71c0 con 0x7fd3d0101de0 2024-09-19T07:32:27.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.674+0000 7fd3d5400640 1 -- 172.21.15.159:0/4078649386 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd3d010e150 con 0x7fd3d0101de0 2024-09-19T07:32:27.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.676+0000 7fd3c7400640 1 -- 172.21.15.159:0/4078649386 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd3c00305e0 con 0x7fd3d0101de0 2024-09-19T07:32:27.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.677+0000 7fd3c7400640 1 --2- 172.21.15.159:0/4078649386 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd3b40a32d0 0x7fd3b40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:27.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.677+0000 7fd3c7400640 1 -- 172.21.15.159:0/4078649386 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fd3c00f0730 con 0x7fd3d0101de0 2024-09-19T07:32:27.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.677+0000 7fd3cde00640 1 --2- 172.21.15.159:0/4078649386 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd3b40a32d0 0x7fd3b40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:27.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.678+0000 7fd3c7400640 1 --2- 172.21.15.159:0/4078649386 >> [v2:172.21.15.189:6800/265696178,v1:172.21.15.189:6801/265696178] conn(0x7fd3b40ae4a0 0x7fd3b40b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:27.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.678+0000 7fd3c7400640 1 -- 172.21.15.159:0/4078649386 --> [v2:172.21.15.189:6800/265696178,v1:172.21.15.189:6801/265696178] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd3c0001d30 con 0x7fd3b40ae4a0 2024-09-19T07:32:27.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.678+0000 7fd3c7400640 1 -- 172.21.15.159:0/4078649386 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fd3c00f0b80 con 0x7fd3d0101de0 2024-09-19T07:32:27.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.678+0000 7fd3cde00640 1 --2- 172.21.15.159:0/4078649386 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd3b40a32d0 0x7fd3b40a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0095f0 tx=0x7fd3bc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:27.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.678+0000 7fd3cf200640 1 --2- 172.21.15.159:0/4078649386 >> [v2:172.21.15.189:6800/265696178,v1:172.21.15.189:6801/265696178] conn(0x7fd3b40ae4a0 0x7fd3b40b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:27.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.679+0000 7fd3cf200640 1 --2- 172.21.15.159:0/4078649386 >> [v2:172.21.15.189:6800/265696178,v1:172.21.15.189:6801/265696178] conn(0x7fd3b40ae4a0 0x7fd3b40b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:27.680 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:27.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.680+0000 7fd3c7400640 1 -- 172.21.15.159:0/4078649386 <== osd.1 v2:172.21.15.189:6800/265696178 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd3c0001d30 con 0x7fd3b40ae4a0 2024-09-19T07:32:27.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.695+0000 7fd3d5400640 1 -- 172.21.15.159:0/4078649386 --> [v2:172.21.15.189:6800/265696178,v1:172.21.15.189:6801/265696178] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd3d01032c0 con 0x7fd3b40ae4a0 2024-09-19T07:32:27.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.695+0000 7fd3c7400640 1 -- 172.21.15.159:0/4078649386 <== osd.1 v2:172.21.15.189:6800/265696178 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd3d01032c0 con 0x7fd3b40ae4a0 2024-09-19T07:32:27.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.695+0000 7fd3d5400640 1 -- 172.21.15.159:0/4078649386 >> [v2:172.21.15.189:6800/265696178,v1:172.21.15.189:6801/265696178] conn(0x7fd3b40ae4a0 msgr2=0x7fd3b40b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:27.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.695+0000 7fd3d5400640 1 --2- 172.21.15.159:0/4078649386 >> [v2:172.21.15.189:6800/265696178,v1:172.21.15.189:6801/265696178] conn(0x7fd3b40ae4a0 0x7fd3b40b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:27.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.695+0000 7fd3d5400640 1 -- 172.21.15.159:0/4078649386 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd3b40a32d0 msgr2=0x7fd3b40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:27.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.695+0000 7fd3d5400640 1 --2- 172.21.15.159:0/4078649386 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd3b40a32d0 0x7fd3b40a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0095f0 tx=0x7fd3bc005f90 comp rx=0 tx=0).stop 2024-09-19T07:32:27.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.695+0000 7fd3d5400640 1 -- 172.21.15.159:0/4078649386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d0101de0 msgr2=0x7fd3d019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:27.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.695+0000 7fd3d5400640 1 --2- 172.21.15.159:0/4078649386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d0101de0 0x7fd3d019a640 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0035040 tx=0x7fd3c0033b60 comp rx=0 tx=0).stop 2024-09-19T07:32:27.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.695+0000 7fd3cf200640 1 -- 172.21.15.159:0/4078649386 reap_dead start 2024-09-19T07:32:27.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.695+0000 7fd3d5400640 1 -- 172.21.15.159:0/4078649386 shutdown_connections 2024-09-19T07:32:27.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.695+0000 7fd3d5400640 1 -- 172.21.15.159:0/4078649386 >> 172.21.15.159:0/4078649386 conn(0x7fd3d00fd8c0 msgr2=0x7fd3d00fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:27.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.696+0000 7fd3d5400640 1 -- 172.21.15.159:0/4078649386 shutdown_connections 2024-09-19T07:32:27.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.696+0000 7fd3d5400640 1 -- 172.21.15.159:0/4078649386 wait complete. 2024-09-19T07:32:28.116 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:27 smithi159 ceph-mon[30472]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-19T07:32:28.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:27 smithi203 ceph-mon[35335]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-19T07:32:28.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:27 smithi189 ceph-mon[34851]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-19T07:32:28.717 INFO:teuthology.orchestra.run.smithi159.stdout:55834574933 2024-09-19T07:32:28.718 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T07:32:29.903 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:29 smithi159 ceph-mon[30472]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-09-19T07:32:30.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:29 smithi203 ceph-mon[35335]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-09-19T07:32:30.287 INFO:teuthology.orchestra.run.smithi159.stdout:347892351007 2024-09-19T07:32:30.287 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-19T07:32:30.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:29 smithi189 ceph-mon[34851]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-09-19T07:32:30.923 INFO:teuthology.orchestra.run.smithi159.stdout:77309411409 2024-09-19T07:32:30.924 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T07:32:31.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-19T07:32:31.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-19T07:32:31.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:30 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-19T07:32:31.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.127+0000 7fab8ee00640 1 -- 172.21.15.159:0/330287188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab88074590 msgr2=0x7fab88074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:31.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.127+0000 7fab8ee00640 1 --2- 172.21.15.159:0/330287188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab88074590 0x7fab88074970 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fab78009900 tx=0x7fab7802a480 comp rx=0 tx=0).stop 2024-09-19T07:32:31.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.127+0000 7fab8ee00640 1 -- 172.21.15.159:0/330287188 shutdown_connections 2024-09-19T07:32:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.127+0000 7fab8ee00640 1 --2- 172.21.15.159:0/330287188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab88112b90 0x7fab88114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.127+0000 7fab8ee00640 1 --2- 172.21.15.159:0/330287188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab88074eb0 0x7fab88112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.127+0000 7fab8ee00640 1 --2- 172.21.15.159:0/330287188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab88074590 0x7fab88074970 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.127+0000 7fab8ee00640 1 -- 172.21.15.159:0/330287188 >> 172.21.15.159:0/330287188 conn(0x7fab880fd8c0 msgr2=0x7fab880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.127+0000 7fab8ee00640 1 -- 172.21.15.159:0/330287188 shutdown_connections 2024-09-19T07:32:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.127+0000 7fab8ee00640 1 -- 172.21.15.159:0/330287188 wait complete. 2024-09-19T07:32:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.128+0000 7fab8ee00640 1 Processor -- start 2024-09-19T07:32:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.128+0000 7fab8ee00640 1 -- start start 2024-09-19T07:32:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.129+0000 7fab8ee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab88074590 0x7fab881a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.129+0000 7fab8ee00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab88074eb0 0x7fab881a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.129+0000 7fab8ee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab88112b90 0x7fab8819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.129+0000 7fab8ee00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fab88118570 con 0x7fab88074590 2024-09-19T07:32:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.129+0000 7fab8ee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fab881183f0 con 0x7fab88112b90 2024-09-19T07:32:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.129+0000 7fab8ee00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fab881186f0 con 0x7fab88074eb0 2024-09-19T07:32:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.129+0000 7fab87e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab88074590 0x7fab881a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.129+0000 7fab87e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab88074590 0x7fab881a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36246/0 (socket says 172.21.15.159:36246) 2024-09-19T07:32:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.129+0000 7fab87e00640 1 -- 172.21.15.159:0/3423992755 learned_addr learned my addr 172.21.15.159:0/3423992755 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.129+0000 7fab8ce00640 1 --2- 172.21.15.159:0/3423992755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab88112b90 0x7fab8819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.129+0000 7fab87400640 1 --2- 172.21.15.159:0/3423992755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab88074eb0 0x7fab881a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.129+0000 7fab87e00640 1 -- 172.21.15.159:0/3423992755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab88074eb0 msgr2=0x7fab881a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.129+0000 7fab87e00640 1 --2- 172.21.15.159:0/3423992755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab88074eb0 0x7fab881a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.129+0000 7fab87e00640 1 -- 172.21.15.159:0/3423992755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab88112b90 msgr2=0x7fab8819e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.129+0000 7fab87e00640 1 --2- 172.21.15.159:0/3423992755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab88112b90 0x7fab8819e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.129+0000 7fab87e00640 1 -- 172.21.15.159:0/3423992755 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8819f010 con 0x7fab88074590 2024-09-19T07:32:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.129+0000 7fab8ce00640 1 --2- 172.21.15.159:0/3423992755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab88112b90 0x7fab8819e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:32:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.130+0000 7fab87e00640 1 --2- 172.21.15.159:0/3423992755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab88074590 0x7fab881a4720 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fab78006010 tx=0x7fab78004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.130+0000 7fab84c00640 1 -- 172.21.15.159:0/3423992755 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7803f030 con 0x7fab88074590 2024-09-19T07:32:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.130+0000 7fab84c00640 1 -- 172.21.15.159:0/3423992755 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab78004440 con 0x7fab88074590 2024-09-19T07:32:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.130+0000 7fab8ee00640 1 -- 172.21.15.159:0/3423992755 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8819f2a0 con 0x7fab88074590 2024-09-19T07:32:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.130+0000 7fab84c00640 1 -- 172.21.15.159:0/3423992755 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78035070 con 0x7fab88074590 2024-09-19T07:32:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.130+0000 7fab8ee00640 1 -- 172.21.15.159:0/3423992755 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab881ab5f0 con 0x7fab88074590 2024-09-19T07:32:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.131+0000 7fab8ee00640 1 -- 172.21.15.159:0/3423992755 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fab880ffae0 con 0x7fab88074590 2024-09-19T07:32:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.133+0000 7fab84c00640 1 -- 172.21.15.159:0/3423992755 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fab78042030 con 0x7fab88074590 2024-09-19T07:32:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.134+0000 7fab84c00640 1 --2- 172.21.15.159:0/3423992755 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fab6c0a32d0 0x7fab6c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.135+0000 7fab84c00640 1 -- 172.21.15.159:0/3423992755 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fab780f0a10 con 0x7fab88074590 2024-09-19T07:32:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.135+0000 7fab87400640 1 --2- 172.21.15.159:0/3423992755 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fab6c0a32d0 0x7fab6c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.135+0000 7fab84c00640 1 --2- 172.21.15.159:0/3423992755 >> [v2:172.21.15.189:6816/3936826456,v1:172.21.15.189:6817/3936826456] conn(0x7fab6c0ae4a0 0x7fab6c0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.135+0000 7fab84c00640 1 -- 172.21.15.159:0/3423992755 --> [v2:172.21.15.189:6816/3936826456,v1:172.21.15.189:6817/3936826456] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fab6c0b1010 con 0x7fab6c0ae4a0 2024-09-19T07:32:31.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.135+0000 7fab84c00640 1 -- 172.21.15.159:0/3423992755 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fab780f0dc0 con 0x7fab88074590 2024-09-19T07:32:31.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.135+0000 7fab8ce00640 1 --2- 172.21.15.159:0/3423992755 >> [v2:172.21.15.189:6816/3936826456,v1:172.21.15.189:6817/3936826456] conn(0x7fab6c0ae4a0 0x7fab6c0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:31.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.135+0000 7fab87400640 1 --2- 172.21.15.159:0/3423992755 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fab6c0a32d0 0x7fab6c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fab88114c20 tx=0x7fab74005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:31.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.136+0000 7fab8ce00640 1 --2- 172.21.15.159:0/3423992755 >> [v2:172.21.15.189:6816/3936826456,v1:172.21.15.189:6817/3936826456] conn(0x7fab6c0ae4a0 0x7fab6c0b0940 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:31.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.137+0000 7fab84c00640 1 -- 172.21.15.159:0/3423992755 <== osd.7 v2:172.21.15.189:6816/3936826456 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fab6c0b1010 con 0x7fab6c0ae4a0 2024-09-19T07:32:31.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.151+0000 7fab8ee00640 1 -- 172.21.15.159:0/3423992755 --> [v2:172.21.15.189:6816/3936826456,v1:172.21.15.189:6817/3936826456] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fab88075cd0 con 0x7fab6c0ae4a0 2024-09-19T07:32:31.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.152+0000 7fab84c00640 1 -- 172.21.15.159:0/3423992755 <== osd.7 v2:172.21.15.189:6816/3936826456 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fab88075cd0 con 0x7fab6c0ae4a0 2024-09-19T07:32:31.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.153+0000 7fab8ee00640 1 -- 172.21.15.159:0/3423992755 >> [v2:172.21.15.189:6816/3936826456,v1:172.21.15.189:6817/3936826456] conn(0x7fab6c0ae4a0 msgr2=0x7fab6c0b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:31.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.153+0000 7fab8ee00640 1 --2- 172.21.15.159:0/3423992755 >> [v2:172.21.15.189:6816/3936826456,v1:172.21.15.189:6817/3936826456] conn(0x7fab6c0ae4a0 0x7fab6c0b0940 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:31.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.153+0000 7fab8ee00640 1 -- 172.21.15.159:0/3423992755 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fab6c0a32d0 msgr2=0x7fab6c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:31.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.153+0000 7fab8ee00640 1 --2- 172.21.15.159:0/3423992755 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fab6c0a32d0 0x7fab6c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fab88114c20 tx=0x7fab74005f90 comp rx=0 tx=0).stop 2024-09-19T07:32:31.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.153+0000 7fab8ee00640 1 -- 172.21.15.159:0/3423992755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab88074590 msgr2=0x7fab881a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:31.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.153+0000 7fab8ee00640 1 --2- 172.21.15.159:0/3423992755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab88074590 0x7fab881a4720 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fab78006010 tx=0x7fab78004040 comp rx=0 tx=0).stop 2024-09-19T07:32:31.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.153+0000 7fab8ce00640 1 -- 172.21.15.159:0/3423992755 reap_dead start 2024-09-19T07:32:31.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.153+0000 7fab8ee00640 1 -- 172.21.15.159:0/3423992755 shutdown_connections 2024-09-19T07:32:31.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.153+0000 7fab8ee00640 1 -- 172.21.15.159:0/3423992755 >> 172.21.15.159:0/3423992755 conn(0x7fab880fd8c0 msgr2=0x7fab880ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:31.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.153+0000 7fab8ee00640 1 -- 172.21.15.159:0/3423992755 shutdown_connections 2024-09-19T07:32:31.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:27.153+0000 7fab8ee00640 1 -- 172.21.15.159:0/3423992755 wait complete. 2024-09-19T07:32:31.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-19T07:32:31.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-19T07:32:31.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:30 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-19T07:32:31.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-19T07:32:31.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-19T07:32:31.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:30 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-19T07:32:31.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:31 smithi159 ceph-mon[30472]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-19T07:32:31.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-19T07:32:31.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-19T07:32:31.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:31 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-19T07:32:31.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:31 smithi159 ceph-mon[30472]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T07:32:32.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:31 smithi203 ceph-mon[35335]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-19T07:32:32.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-19T07:32:32.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-19T07:32:32.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:31 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-19T07:32:32.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:31 smithi203 ceph-mon[35335]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T07:32:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:31 smithi189 ceph-mon[34851]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-19T07:32:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-19T07:32:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-19T07:32:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:31 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-19T07:32:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:31 smithi189 ceph-mon[34851]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T07:32:32.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.504+0000 7f4632c00640 1 -- 172.21.15.159:0/2558917704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f462c102b90 msgr2=0x7f462c102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:32.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.504+0000 7f4632c00640 1 --2- 172.21.15.159:0/2558917704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f462c102b90 0x7f462c102f70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4620009900 tx=0x7f462002a480 comp rx=0 tx=0).stop 2024-09-19T07:32:32.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.504+0000 7f4632c00640 1 -- 172.21.15.159:0/2558917704 shutdown_connections 2024-09-19T07:32:32.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.504+0000 7f4632c00640 1 --2- 172.21.15.159:0/2558917704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f462c10bfc0 0x7f462c10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:32.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.504+0000 7f4632c00640 1 --2- 172.21.15.159:0/2558917704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f462c1034b0 0x7f462c10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:32.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.504+0000 7f4632c00640 1 --2- 172.21.15.159:0/2558917704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f462c102b90 0x7f462c102f70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:32.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.504+0000 7f4632c00640 1 -- 172.21.15.159:0/2558917704 >> 172.21.15.159:0/2558917704 conn(0x7f462c0fc610 msgr2=0x7f462c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:32.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.504+0000 7f4632c00640 1 -- 172.21.15.159:0/2558917704 shutdown_connections 2024-09-19T07:32:32.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.504+0000 7f4632c00640 1 -- 172.21.15.159:0/2558917704 wait complete. 2024-09-19T07:32:32.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.505+0000 7f4632c00640 1 Processor -- start 2024-09-19T07:32:32.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.505+0000 7f4632c00640 1 -- start start 2024-09-19T07:32:32.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.506+0000 7f4632c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f462c102b90 0x7f462c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:32.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.506+0000 7f4632c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f462c1034b0 0x7f462c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:32.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.506+0000 7f4632c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f462c10bfc0 0x7f462c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:32.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.506+0000 7f4632c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f462c111dc0 con 0x7f462c10bfc0 2024-09-19T07:32:32.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.506+0000 7f4632c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f462c111c40 con 0x7f462c102b90 2024-09-19T07:32:32.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.506+0000 7f4632c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f462c111f40 con 0x7f462c1034b0 2024-09-19T07:32:32.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.506+0000 7f4630e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f462c1034b0 0x7f462c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:32.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.506+0000 7f4630e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f462c1034b0 0x7f462c19aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:59160/0 (socket says 172.21.15.159:59160) 2024-09-19T07:32:32.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.506+0000 7f4630e00640 1 -- 172.21.15.159:0/1386074956 learned_addr learned my addr 172.21.15.159:0/1386074956 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:32.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.506+0000 7f4631800640 1 --2- 172.21.15.159:0/1386074956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f462c102b90 0x7f462c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:32.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.506+0000 7f4632200640 1 --2- 172.21.15.159:0/1386074956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f462c10bfc0 0x7f462c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:32.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.506+0000 7f4630e00640 1 -- 172.21.15.159:0/1386074956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f462c102b90 msgr2=0x7f462c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:32.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.506+0000 7f4630e00640 1 --2- 172.21.15.159:0/1386074956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f462c102b90 0x7f462c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:32.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.506+0000 7f4630e00640 1 -- 172.21.15.159:0/1386074956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f462c10bfc0 msgr2=0x7f462c19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:32.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.506+0000 7f4630e00640 1 --2- 172.21.15.159:0/1386074956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f462c10bfc0 0x7f462c19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:32.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.507+0000 7f4630e00640 1 -- 172.21.15.159:0/1386074956 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c19f4f0 con 0x7f462c1034b0 2024-09-19T07:32:32.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.507+0000 7f4631800640 1 --2- 172.21.15.159:0/1386074956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f462c102b90 0x7f462c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:32:32.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.507+0000 7f4630e00640 1 --2- 172.21.15.159:0/1386074956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f462c1034b0 0x7f462c19aa80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f461800d950 tx=0x7f461800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:32.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.507+0000 7f462a000640 1 -- 172.21.15.159:0/1386074956 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618017030 con 0x7f462c1034b0 2024-09-19T07:32:32.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.507+0000 7f462a000640 1 -- 172.21.15.159:0/1386074956 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4618004020 con 0x7f462c1034b0 2024-09-19T07:32:32.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.507+0000 7f4632c00640 1 -- 172.21.15.159:0/1386074956 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462c19f7e0 con 0x7f462c1034b0 2024-09-19T07:32:32.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.507+0000 7f462a000640 1 -- 172.21.15.159:0/1386074956 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46180047a0 con 0x7f462c1034b0 2024-09-19T07:32:32.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.507+0000 7f4632c00640 1 -- 172.21.15.159:0/1386074956 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462c1a70c0 con 0x7f462c1034b0 2024-09-19T07:32:32.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.508+0000 7f4632c00640 1 -- 172.21.15.159:0/1386074956 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f462c0fec00 con 0x7f462c1034b0 2024-09-19T07:32:32.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.511+0000 7f462a000640 1 -- 172.21.15.159:0/1386074956 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f46180041d0 con 0x7f462c1034b0 2024-09-19T07:32:32.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.512+0000 7f462a000640 1 --2- 172.21.15.159:0/1386074956 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f46140a3350 0x7f46140a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:32.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.512+0000 7f462a000640 1 -- 172.21.15.159:0/1386074956 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f46180cff00 con 0x7f462c1034b0 2024-09-19T07:32:32.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.513+0000 7f4631800640 1 --2- 172.21.15.159:0/1386074956 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f46140a3350 0x7f46140a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:32.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.513+0000 7f462a000640 1 --2- 172.21.15.159:0/1386074956 >> [v2:172.21.15.189:6808/1036888180,v1:172.21.15.189:6809/1036888180] conn(0x7f46140ae520 0x7f46140b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:32.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.513+0000 7f462a000640 1 -- 172.21.15.159:0/1386074956 --> [v2:172.21.15.189:6808/1036888180,v1:172.21.15.189:6809/1036888180] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f46140b1090 con 0x7f46140ae520 2024-09-19T07:32:32.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.513+0000 7f462a000640 1 -- 172.21.15.159:0/1386074956 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f46180d02b0 con 0x7f462c1034b0 2024-09-19T07:32:32.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.513+0000 7f4632200640 1 --2- 172.21.15.159:0/1386074956 >> [v2:172.21.15.189:6808/1036888180,v1:172.21.15.189:6809/1036888180] conn(0x7f46140ae520 0x7f46140b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:32.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.513+0000 7f4631800640 1 --2- 172.21.15.159:0/1386074956 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f46140a3350 0x7f46140a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4620005ab0 tx=0x7f4620005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:32.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.514+0000 7f4632200640 1 --2- 172.21.15.159:0/1386074956 >> [v2:172.21.15.189:6808/1036888180,v1:172.21.15.189:6809/1036888180] conn(0x7f46140ae520 0x7f46140b09c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:32.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.514+0000 7f462a000640 1 -- 172.21.15.159:0/1386074956 <== osd.4 v2:172.21.15.189:6808/1036888180 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f46140b1090 con 0x7f46140ae520 2024-09-19T07:32:32.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.529+0000 7f4632c00640 1 -- 172.21.15.159:0/1386074956 --> [v2:172.21.15.189:6808/1036888180,v1:172.21.15.189:6809/1036888180] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f462c06a850 con 0x7f46140ae520 2024-09-19T07:32:32.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.529+0000 7f462a000640 1 -- 172.21.15.159:0/1386074956 <== osd.4 v2:172.21.15.189:6808/1036888180 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f462c06a850 con 0x7f46140ae520 2024-09-19T07:32:32.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.530+0000 7f4632c00640 1 -- 172.21.15.159:0/1386074956 >> [v2:172.21.15.189:6808/1036888180,v1:172.21.15.189:6809/1036888180] conn(0x7f46140ae520 msgr2=0x7f46140b09c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:32.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.530+0000 7f4632c00640 1 --2- 172.21.15.159:0/1386074956 >> [v2:172.21.15.189:6808/1036888180,v1:172.21.15.189:6809/1036888180] conn(0x7f46140ae520 0x7f46140b09c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:32.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.530+0000 7f4632c00640 1 -- 172.21.15.159:0/1386074956 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f46140a3350 msgr2=0x7f46140a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:32.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.530+0000 7f4632c00640 1 --2- 172.21.15.159:0/1386074956 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f46140a3350 0x7f46140a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4620005ab0 tx=0x7f4620005a20 comp rx=0 tx=0).stop 2024-09-19T07:32:32.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.530+0000 7f4632c00640 1 -- 172.21.15.159:0/1386074956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f462c1034b0 msgr2=0x7f462c19aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:32.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.530+0000 7f4632c00640 1 --2- 172.21.15.159:0/1386074956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f462c1034b0 0x7f462c19aa80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f461800d950 tx=0x7f461800de10 comp rx=0 tx=0).stop 2024-09-19T07:32:32.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.530+0000 7f4632200640 1 -- 172.21.15.159:0/1386074956 reap_dead start 2024-09-19T07:32:32.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.530+0000 7f4632c00640 1 -- 172.21.15.159:0/1386074956 shutdown_connections 2024-09-19T07:32:32.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.530+0000 7f4632c00640 1 -- 172.21.15.159:0/1386074956 >> 172.21.15.159:0/1386074956 conn(0x7f462c0fc610 msgr2=0x7f462c0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:32.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.530+0000 7f4632c00640 1 -- 172.21.15.159:0/1386074956 shutdown_connections 2024-09-19T07:32:32.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:30.531+0000 7f4632c00640 1 -- 172.21.15.159:0/1386074956 wait complete. 2024-09-19T07:32:33.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:32 smithi159 ceph-mon[30472]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T07:32:33.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:32 smithi203 ceph-mon[35335]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T07:32:33.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:32 smithi189 ceph-mon[34851]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T07:32:33.443 INFO:teuthology.orchestra.run.smithi159.stdout:223338299445 2024-09-19T07:32:33.443 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T07:32:33.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.414+0000 7f6a28200640 1 -- 172.21.15.159:0/4020825655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a201049a0 msgr2=0x7f6a201089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:33.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.414+0000 7f6a28200640 1 --2- 172.21.15.159:0/4020825655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a201049a0 0x7f6a201089f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009900 tx=0x7f6a0c02a410 comp rx=0 tx=0).stop 2024-09-19T07:32:33.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.415+0000 7f6a28200640 1 -- 172.21.15.159:0/4020825655 shutdown_connections 2024-09-19T07:32:33.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.415+0000 7f6a28200640 1 --2- 172.21.15.159:0/4020825655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a20109120 0x7f6a20110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:33.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.415+0000 7f6a28200640 1 --2- 172.21.15.159:0/4020825655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a201049a0 0x7f6a201089f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:33.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.415+0000 7f6a28200640 1 --2- 172.21.15.159:0/4020825655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a20103ff0 0x7f6a201043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:33.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.415+0000 7f6a28200640 1 -- 172.21.15.159:0/4020825655 >> 172.21.15.159:0/4020825655 conn(0x7f6a200ffa90 msgr2=0x7f6a20101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:33.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.415+0000 7f6a28200640 1 -- 172.21.15.159:0/4020825655 shutdown_connections 2024-09-19T07:32:33.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.415+0000 7f6a28200640 1 -- 172.21.15.159:0/4020825655 wait complete. 2024-09-19T07:32:33.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.416+0000 7f6a28200640 1 Processor -- start 2024-09-19T07:32:33.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.416+0000 7f6a28200640 1 -- start start 2024-09-19T07:32:33.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.416+0000 7f6a28200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a20103ff0 0x7f6a2019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:33.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.416+0000 7f6a28200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a201049a0 0x7f6a2019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:33.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.416+0000 7f6a28200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a20109120 0x7f6a2019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:33.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.416+0000 7f6a28200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6a20114120 con 0x7f6a20109120 2024-09-19T07:32:33.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.416+0000 7f6a28200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a20113fa0 con 0x7f6a201049a0 2024-09-19T07:32:33.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.416+0000 7f6a28200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6a201142a0 con 0x7f6a20103ff0 2024-09-19T07:32:33.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.417+0000 7f6a26200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a20109120 0x7f6a2019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:33.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.417+0000 7f6a24e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a201049a0 0x7f6a2019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:33.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.417+0000 7f6a24e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a201049a0 0x7f6a2019abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:39530/0 (socket says 172.21.15.159:39530) 2024-09-19T07:32:33.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.417+0000 7f6a26200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a20109120 0x7f6a2019ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37234/0 (socket says 172.21.15.159:37234) 2024-09-19T07:32:33.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.417+0000 7f6a24e00640 1 -- 172.21.15.159:0/2581813683 learned_addr learned my addr 172.21.15.159:0/2581813683 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:33.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.417+0000 7f6a25800640 1 --2- 172.21.15.159:0/2581813683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a20103ff0 0x7f6a2019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:33.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.417+0000 7f6a26200640 1 -- 172.21.15.159:0/2581813683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a20103ff0 msgr2=0x7f6a2019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:33.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.417+0000 7f6a26200640 1 --2- 172.21.15.159:0/2581813683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a20103ff0 0x7f6a2019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:33.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.417+0000 7f6a26200640 1 -- 172.21.15.159:0/2581813683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a201049a0 msgr2=0x7f6a2019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:33.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.417+0000 7f6a26200640 1 --2- 172.21.15.159:0/2581813683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a201049a0 0x7f6a2019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:33.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.417+0000 7f6a26200640 1 -- 172.21.15.159:0/2581813683 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2019f650 con 0x7f6a20109120 2024-09-19T07:32:33.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.418+0000 7f6a26200640 1 --2- 172.21.15.159:0/2581813683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a20109120 0x7f6a2019ef70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009fd0 tx=0x7f6a1800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:33.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.418+0000 7f6a1e000640 1 -- 172.21.15.159:0/2581813683 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1800ce20 con 0x7f6a20109120 2024-09-19T07:32:33.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.418+0000 7f6a28200640 1 -- 172.21.15.159:0/2581813683 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2019f940 con 0x7f6a20109120 2024-09-19T07:32:33.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.418+0000 7f6a1e000640 1 -- 172.21.15.159:0/2581813683 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a18005020 con 0x7f6a20109120 2024-09-19T07:32:33.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.418+0000 7f6a28200640 1 -- 172.21.15.159:0/2581813683 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a201a7220 con 0x7f6a20109120 2024-09-19T07:32:33.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.419+0000 7f6a1e000640 1 -- 172.21.15.159:0/2581813683 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18010700 con 0x7f6a20109120 2024-09-19T07:32:33.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.419+0000 7f6a28200640 1 -- 172.21.15.159:0/2581813683 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6a20062900 con 0x7f6a20109120 2024-09-19T07:32:33.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.421+0000 7f6a1e000640 1 -- 172.21.15.159:0/2581813683 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6a18005540 con 0x7f6a20109120 2024-09-19T07:32:33.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.422+0000 7f6a1e000640 1 --2- 172.21.15.159:0/2581813683 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6a080a32d0 0x7f6a080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:33.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.422+0000 7f6a1e000640 1 -- 172.21.15.159:0/2581813683 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f6a180d0450 con 0x7f6a20109120 2024-09-19T07:32:33.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.422+0000 7f6a25800640 1 --2- 172.21.15.159:0/2581813683 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6a080a32d0 0x7f6a080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:33.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.423+0000 7f6a1e000640 1 --2- 172.21.15.159:0/2581813683 >> [v2:172.21.15.159:6802/2738900830,v1:172.21.15.159:6803/2738900830] conn(0x7f6a080ae620 0x7f6a080b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:33.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.423+0000 7f6a1e000640 1 -- 172.21.15.159:0/2581813683 --> [v2:172.21.15.159:6802/2738900830,v1:172.21.15.159:6803/2738900830] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a18004810 con 0x7f6a080ae620 2024-09-19T07:32:33.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.423+0000 7f6a1e000640 1 -- 172.21.15.159:0/2581813683 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f6a180d7050 con 0x7f6a20109120 2024-09-19T07:32:33.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.423+0000 7f6a24e00640 1 --2- 172.21.15.159:0/2581813683 >> [v2:172.21.15.159:6802/2738900830,v1:172.21.15.159:6803/2738900830] conn(0x7f6a080ae620 0x7f6a080b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:33.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.423+0000 7f6a25800640 1 --2- 172.21.15.159:0/2581813683 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6a080a32d0 0x7f6a080a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009bd0 tx=0x7f6a14005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:33.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.424+0000 7f6a24e00640 1 --2- 172.21.15.159:0/2581813683 >> [v2:172.21.15.159:6802/2738900830,v1:172.21.15.159:6803/2738900830] conn(0x7f6a080ae620 0x7f6a080b0ac0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:33.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.425+0000 7f6a1e000640 1 -- 172.21.15.159:0/2581813683 <== osd.0 v2:172.21.15.159:6802/2738900830 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6a18004810 con 0x7f6a080ae620 2024-09-19T07:32:33.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.439+0000 7f6a28200640 1 -- 172.21.15.159:0/2581813683 --> [v2:172.21.15.159:6802/2738900830,v1:172.21.15.159:6803/2738900830] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6a20062d80 con 0x7f6a080ae620 2024-09-19T07:32:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.440+0000 7f6a1e000640 1 -- 172.21.15.159:0/2581813683 <== osd.0 v2:172.21.15.159:6802/2738900830 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6a20062d80 con 0x7f6a080ae620 2024-09-19T07:32:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.440+0000 7f6a28200640 1 -- 172.21.15.159:0/2581813683 >> [v2:172.21.15.159:6802/2738900830,v1:172.21.15.159:6803/2738900830] conn(0x7f6a080ae620 msgr2=0x7f6a080b0ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.440+0000 7f6a28200640 1 --2- 172.21.15.159:0/2581813683 >> [v2:172.21.15.159:6802/2738900830,v1:172.21.15.159:6803/2738900830] conn(0x7f6a080ae620 0x7f6a080b0ac0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.441+0000 7f6a28200640 1 -- 172.21.15.159:0/2581813683 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6a080a32d0 msgr2=0x7f6a080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.441+0000 7f6a28200640 1 --2- 172.21.15.159:0/2581813683 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f6a080a32d0 0x7f6a080a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009bd0 tx=0x7f6a14005f90 comp rx=0 tx=0).stop 2024-09-19T07:32:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.441+0000 7f6a28200640 1 -- 172.21.15.159:0/2581813683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a20109120 msgr2=0x7f6a2019ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.441+0000 7f6a28200640 1 --2- 172.21.15.159:0/2581813683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a20109120 0x7f6a2019ef70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009fd0 tx=0x7f6a1800ef10 comp rx=0 tx=0).stop 2024-09-19T07:32:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.441+0000 7f6a26200640 1 -- 172.21.15.159:0/2581813683 reap_dead start 2024-09-19T07:32:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.441+0000 7f6a28200640 1 -- 172.21.15.159:0/2581813683 shutdown_connections 2024-09-19T07:32:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.441+0000 7f6a28200640 1 -- 172.21.15.159:0/2581813683 >> 172.21.15.159:0/2581813683 conn(0x7f6a200ffa90 msgr2=0x7f6a20100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.441+0000 7f6a28200640 1 -- 172.21.15.159:0/2581813683 shutdown_connections 2024-09-19T07:32:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:33.441+0000 7f6a28200640 1 -- 172.21.15.159:0/2581813683 wait complete. 2024-09-19T07:32:34.129 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:33 smithi159 ceph-mon[30472]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:32:34.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:33 smithi203 ceph-mon[35335]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:32:34.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:33 smithi189 ceph-mon[34851]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:32:35.183 INFO:teuthology.orchestra.run.smithi159.stdout:146028888134 2024-09-19T07:32:35.184 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T07:32:35.314 INFO:teuthology.orchestra.run.smithi159.stdout:317827579941 2024-09-19T07:32:35.314 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-19T07:32:35.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.930+0000 7f5da4c00640 1 -- 172.21.15.159:0/3845504317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da0076720 msgr2=0x7f5da0076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:35.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.930+0000 7f5da4c00640 1 --2- 172.21.15.159:0/3845504317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da0076720 0x7f5da0076b00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009900 tx=0x7f5d9002a410 comp rx=0 tx=0).stop 2024-09-19T07:32:35.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.930+0000 7f5da4c00640 1 -- 172.21.15.159:0/3845504317 shutdown_connections 2024-09-19T07:32:35.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.930+0000 7f5da4c00640 1 --2- 172.21.15.159:0/3845504317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5da0112870 0x7f5da0114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:35.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.930+0000 7f5da4c00640 1 --2- 172.21.15.159:0/3845504317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5da0077040 0x7f5da0112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:35.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.930+0000 7f5da4c00640 1 --2- 172.21.15.159:0/3845504317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da0076720 0x7f5da0076b00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:35.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.930+0000 7f5da4c00640 1 -- 172.21.15.159:0/3845504317 >> 172.21.15.159:0/3845504317 conn(0x7f5da00ff8d0 msgr2=0x7f5da0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:35.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.930+0000 7f5da4c00640 1 -- 172.21.15.159:0/3845504317 shutdown_connections 2024-09-19T07:32:35.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.930+0000 7f5da4c00640 1 -- 172.21.15.159:0/3845504317 wait complete. 2024-09-19T07:32:35.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.931+0000 7f5da4c00640 1 Processor -- start 2024-09-19T07:32:35.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.932+0000 7f5da4c00640 1 -- start start 2024-09-19T07:32:35.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.932+0000 7f5da4c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da0076720 0x7f5da01a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:35.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.932+0000 7f5da4c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5da0077040 0x7f5da01a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:35.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.932+0000 7f5da4c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5da0112870 0x7f5da019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:35.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.932+0000 7f5da4c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5da0118450 con 0x7f5da0076720 2024-09-19T07:32:35.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.932+0000 7f5da4c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01182d0 con 0x7f5da0077040 2024-09-19T07:32:35.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.932+0000 7f5da4c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01185d0 con 0x7f5da0112870 2024-09-19T07:32:35.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.932+0000 7f5d9f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da0076720 0x7f5da01a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:35.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.933+0000 7f5d9ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5da0077040 0x7f5da01a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:35.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.933+0000 7f5d9f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da0076720 0x7f5da01a4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36296/0 (socket says 172.21.15.159:36296) 2024-09-19T07:32:35.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.933+0000 7f5d9ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5da0077040 0x7f5da01a4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:52092/0 (socket says 172.21.15.159:52092) 2024-09-19T07:32:35.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.933+0000 7f5d9f400640 1 -- 172.21.15.159:0/343466263 learned_addr learned my addr 172.21.15.159:0/343466263 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:35.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.933+0000 7f5d9fe00640 1 --2- 172.21.15.159:0/343466263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5da0112870 0x7f5da019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:35.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.933+0000 7f5d9ea00640 1 -- 172.21.15.159:0/343466263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5da0112870 msgr2=0x7f5da019e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:35.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.933+0000 7f5d9ea00640 1 --2- 172.21.15.159:0/343466263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5da0112870 0x7f5da019e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:35.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.933+0000 7f5d9ea00640 1 -- 172.21.15.159:0/343466263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da0076720 msgr2=0x7f5da01a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:35.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.933+0000 7f5d9ea00640 1 --2- 172.21.15.159:0/343466263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da0076720 0x7f5da01a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:35.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.933+0000 7f5d9ea00640 1 -- 172.21.15.159:0/343466263 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da019efb0 con 0x7f5da0077040 2024-09-19T07:32:35.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.933+0000 7f5d9f400640 1 --2- 172.21.15.159:0/343466263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da0076720 0x7f5da01a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:32:35.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.933+0000 7f5d9ea00640 1 --2- 172.21.15.159:0/343466263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5da0077040 0x7f5da01a4bb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00e9f0 tx=0x7f5d8c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:35.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.934+0000 7f5d97e00640 1 -- 172.21.15.159:0/343466263 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c00ccc0 con 0x7f5da0077040 2024-09-19T07:32:35.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.934+0000 7f5d97e00640 1 -- 172.21.15.159:0/343466263 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d8c010910 con 0x7f5da0077040 2024-09-19T07:32:35.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.934+0000 7f5d97e00640 1 -- 172.21.15.159:0/343466263 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c018960 con 0x7f5da0077040 2024-09-19T07:32:35.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.934+0000 7f5da4c00640 1 -- 172.21.15.159:0/343466263 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da019f2a0 con 0x7f5da0077040 2024-09-19T07:32:35.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.934+0000 7f5da4c00640 1 -- 172.21.15.159:0/343466263 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da01ab5f0 con 0x7f5da0077040 2024-09-19T07:32:35.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.935+0000 7f5da4c00640 1 -- 172.21.15.159:0/343466263 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5da0062940 con 0x7f5da0077040 2024-09-19T07:32:35.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.937+0000 7f5d97e00640 1 -- 172.21.15.159:0/343466263 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5d8c010ac0 con 0x7f5da0077040 2024-09-19T07:32:35.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.938+0000 7f5d97e00640 1 --2- 172.21.15.159:0/343466263 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f5d840a33a0 0x7f5d840a5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:35.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.938+0000 7f5d97e00640 1 -- 172.21.15.159:0/343466263 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f5d8c0d09d0 con 0x7f5da0077040 2024-09-19T07:32:35.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.938+0000 7f5d9f400640 1 --2- 172.21.15.159:0/343466263 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f5d840a33a0 0x7f5d840a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:35.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.938+0000 7f5d97e00640 1 --2- 172.21.15.159:0/343466263 >> [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578] conn(0x7f5d840ae6f0 0x7f5d840b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:35.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.938+0000 7f5d97e00640 1 -- 172.21.15.159:0/343466263 --> [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d8c028ea0 con 0x7f5d840ae6f0 2024-09-19T07:32:35.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.938+0000 7f5d97e00640 1 -- 172.21.15.159:0/343466263 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f5d8c0d0d80 con 0x7f5da0077040 2024-09-19T07:32:35.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.939+0000 7f5d9fe00640 1 --2- 172.21.15.159:0/343466263 >> [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578] conn(0x7f5d840ae6f0 0x7f5d840b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:35.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.939+0000 7f5d9f400640 1 --2- 172.21.15.159:0/343466263 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f5d840a33a0 0x7f5d840a5860 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5d900096c0 tx=0x7f5d9002a910 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:35.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.939+0000 7f5d9fe00640 1 --2- 172.21.15.159:0/343466263 >> [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578] conn(0x7f5d840ae6f0 0x7f5d840b0b90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:35.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.940+0000 7f5d97e00640 1 -- 172.21.15.159:0/343466263 <== osd.6 v2:172.21.15.159:6818/3817684578 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d8c028ea0 con 0x7f5d840ae6f0 2024-09-19T07:32:35.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.954+0000 7f5da4c00640 1 -- 172.21.15.159:0/343466263 --> [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5da0062d40 con 0x7f5d840ae6f0 2024-09-19T07:32:35.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.955+0000 7f5d97e00640 1 -- 172.21.15.159:0/343466263 <== osd.6 v2:172.21.15.159:6818/3817684578 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5da0062d40 con 0x7f5d840ae6f0 2024-09-19T07:32:35.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.955+0000 7f5da4c00640 1 -- 172.21.15.159:0/343466263 >> [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578] conn(0x7f5d840ae6f0 msgr2=0x7f5d840b0b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:35.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.955+0000 7f5da4c00640 1 --2- 172.21.15.159:0/343466263 >> [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578] conn(0x7f5d840ae6f0 0x7f5d840b0b90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:35.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.955+0000 7f5da4c00640 1 -- 172.21.15.159:0/343466263 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f5d840a33a0 msgr2=0x7f5d840a5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:35.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.955+0000 7f5da4c00640 1 --2- 172.21.15.159:0/343466263 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f5d840a33a0 0x7f5d840a5860 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5d900096c0 tx=0x7f5d9002a910 comp rx=0 tx=0).stop 2024-09-19T07:32:35.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.955+0000 7f5da4c00640 1 -- 172.21.15.159:0/343466263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5da0077040 msgr2=0x7f5da01a4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:35.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.955+0000 7f5da4c00640 1 --2- 172.21.15.159:0/343466263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5da0077040 0x7f5da01a4bb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00e9f0 tx=0x7f5d8c00eeb0 comp rx=0 tx=0).stop 2024-09-19T07:32:35.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.955+0000 7f5d9fe00640 1 -- 172.21.15.159:0/343466263 reap_dead start 2024-09-19T07:32:35.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.955+0000 7f5da4c00640 1 -- 172.21.15.159:0/343466263 shutdown_connections 2024-09-19T07:32:35.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.955+0000 7f5da4c00640 1 -- 172.21.15.159:0/343466263 >> 172.21.15.159:0/343466263 conn(0x7f5da00ff8d0 msgr2=0x7f5da0101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:35.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.955+0000 7f5da4c00640 1 -- 172.21.15.159:0/343466263 shutdown_connections 2024-09-19T07:32:35.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:31.955+0000 7f5da4c00640 1 -- 172.21.15.159:0/343466263 wait complete. 2024-09-19T07:32:35.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.123+0000 7f5047200640 1 -- 172.21.15.159:0/2385606022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5048108f60 msgr2=0x7f5048110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:35.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.123+0000 7f5047200640 1 --2- 172.21.15.159:0/2385606022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5048108f60 0x7f5048110ae0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5040009ca0 tx=0x7f504002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:32:35.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.123+0000 7f5047200640 1 -- 172.21.15.159:0/2385606022 shutdown_connections 2024-09-19T07:32:35.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.123+0000 7f5047200640 1 --2- 172.21.15.159:0/2385606022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5048108f60 0x7f5048110ae0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:35.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.123+0000 7f5047200640 1 --2- 172.21.15.159:0/2385606022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50481047e0 0x7f5048108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:35.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.123+0000 7f5047200640 1 --2- 172.21.15.159:0/2385606022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5048103e30 0x7f5048104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:35.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.123+0000 7f5047200640 1 -- 172.21.15.159:0/2385606022 >> 172.21.15.159:0/2385606022 conn(0x7f50480ff910 msgr2=0x7f5048101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:35.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.123+0000 7f5047200640 1 -- 172.21.15.159:0/2385606022 shutdown_connections 2024-09-19T07:32:35.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.123+0000 7f5047200640 1 -- 172.21.15.159:0/2385606022 wait complete. 2024-09-19T07:32:35.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.124+0000 7f5047200640 1 Processor -- start 2024-09-19T07:32:35.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.124+0000 7f5047200640 1 -- start start 2024-09-19T07:32:35.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.125+0000 7f5047200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5048103e30 0x7f504819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:35.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.125+0000 7f5047200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50481047e0 0x7f504819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:35.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.125+0000 7f5047200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5048108f60 0x7f504819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:35.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.125+0000 7f5047200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5048113fc0 con 0x7f5048108f60 2024-09-19T07:32:35.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.125+0000 7f5047200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5048113e40 con 0x7f5048103e30 2024-09-19T07:32:35.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.125+0000 7f5047200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5048114140 con 0x7f50481047e0 2024-09-19T07:32:35.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.125+0000 7f5045400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50481047e0 0x7f504819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:35.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.125+0000 7f5046800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5048108f60 0x7f504819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:35.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.125+0000 7f5045400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50481047e0 0x7f504819a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:44980/0 (socket says 172.21.15.159:44980) 2024-09-19T07:32:35.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.125+0000 7f5045400640 1 -- 172.21.15.159:0/2221217455 learned_addr learned my addr 172.21.15.159:0/2221217455 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:35.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.125+0000 7f5045e00640 1 --2- 172.21.15.159:0/2221217455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5048103e30 0x7f504819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:35.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.126+0000 7f5046800640 1 -- 172.21.15.159:0/2221217455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50481047e0 msgr2=0x7f504819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:35.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.126+0000 7f5046800640 1 --2- 172.21.15.159:0/2221217455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50481047e0 0x7f504819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:35.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.126+0000 7f5046800640 1 -- 172.21.15.159:0/2221217455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5048103e30 msgr2=0x7f504819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:35.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.126+0000 7f5046800640 1 --2- 172.21.15.159:0/2221217455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5048103e30 0x7f504819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:35.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.126+0000 7f5046800640 1 -- 172.21.15.159:0/2221217455 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504819f3f0 con 0x7f5048108f60 2024-09-19T07:32:35.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.126+0000 7f5045400640 1 --2- 172.21.15.159:0/2221217455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50481047e0 0x7f504819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:32:35.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.126+0000 7f5045e00640 1 --2- 172.21.15.159:0/2221217455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5048103e30 0x7f504819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:32:35.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.126+0000 7f5046800640 1 --2- 172.21.15.159:0/2221217455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5048108f60 0x7f504819ed10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f504002ade0 tx=0x7f50400319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:35.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.126+0000 7f503ea00640 1 -- 172.21.15.159:0/2221217455 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040003b70 con 0x7f5048108f60 2024-09-19T07:32:35.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.126+0000 7f5047200640 1 -- 172.21.15.159:0/2221217455 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504819f680 con 0x7f5048108f60 2024-09-19T07:32:35.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.126+0000 7f503ea00640 1 -- 172.21.15.159:0/2221217455 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5040003d10 con 0x7f5048108f60 2024-09-19T07:32:35.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.126+0000 7f503ea00640 1 -- 172.21.15.159:0/2221217455 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504002fcb0 con 0x7f5048108f60 2024-09-19T07:32:35.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.126+0000 7f5047200640 1 -- 172.21.15.159:0/2221217455 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50481a6fc0 con 0x7f5048108f60 2024-09-19T07:32:35.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.127+0000 7f5047200640 1 -- 172.21.15.159:0/2221217455 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5048062980 con 0x7f5048108f60 2024-09-19T07:32:35.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.129+0000 7f503ea00640 1 -- 172.21.15.159:0/2221217455 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5040004050 con 0x7f5048108f60 2024-09-19T07:32:35.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.130+0000 7f503ea00640 1 --2- 172.21.15.159:0/2221217455 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f502c0a32d0 0x7f502c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:35.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.131+0000 7f503ea00640 1 -- 172.21.15.159:0/2221217455 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f50400f1090 con 0x7f5048108f60 2024-09-19T07:32:35.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.131+0000 7f5045e00640 1 --2- 172.21.15.159:0/2221217455 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f502c0a32d0 0x7f502c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:35.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.131+0000 7f503ea00640 1 --2- 172.21.15.159:0/2221217455 >> [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747] conn(0x7f502c0ae620 0x7f502c0b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:35.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.131+0000 7f503ea00640 1 -- 172.21.15.159:0/2221217455 --> [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f50400020b0 con 0x7f502c0ae620 2024-09-19T07:32:35.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.131+0000 7f503ea00640 1 -- 172.21.15.159:0/2221217455 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f50400f1440 con 0x7f5048108f60 2024-09-19T07:32:35.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.131+0000 7f5045400640 1 --2- 172.21.15.159:0/2221217455 >> [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747] conn(0x7f502c0ae620 0x7f502c0b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:35.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.131+0000 7f5045e00640 1 --2- 172.21.15.159:0/2221217455 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f502c0a32d0 0x7f502c0a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5038009fc0 tx=0x7f5038005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:35.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.132+0000 7f5045400640 1 --2- 172.21.15.159:0/2221217455 >> [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747] conn(0x7f502c0ae620 0x7f502c0b0ac0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:35.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.133+0000 7f503ea00640 1 -- 172.21.15.159:0/2221217455 <== osd.9 v2:172.21.15.159:6826/4185848747 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f50400020b0 con 0x7f502c0ae620 2024-09-19T07:32:35.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.148+0000 7f5047200640 1 -- 172.21.15.159:0/2221217455 --> [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5048062d80 con 0x7f502c0ae620 2024-09-19T07:32:35.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.149+0000 7f503ea00640 1 -- 172.21.15.159:0/2221217455 <== osd.9 v2:172.21.15.159:6826/4185848747 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5048062d80 con 0x7f502c0ae620 2024-09-19T07:32:35.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.150+0000 7f5047200640 1 -- 172.21.15.159:0/2221217455 >> [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747] conn(0x7f502c0ae620 msgr2=0x7f502c0b0ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:35.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.150+0000 7f5047200640 1 --2- 172.21.15.159:0/2221217455 >> [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747] conn(0x7f502c0ae620 0x7f502c0b0ac0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:35.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.150+0000 7f5047200640 1 -- 172.21.15.159:0/2221217455 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f502c0a32d0 msgr2=0x7f502c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:35.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.150+0000 7f5047200640 1 --2- 172.21.15.159:0/2221217455 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f502c0a32d0 0x7f502c0a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5038009fc0 tx=0x7f5038005f90 comp rx=0 tx=0).stop 2024-09-19T07:32:35.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.150+0000 7f5047200640 1 -- 172.21.15.159:0/2221217455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5048108f60 msgr2=0x7f504819ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:35.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.150+0000 7f5047200640 1 --2- 172.21.15.159:0/2221217455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5048108f60 0x7f504819ed10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f504002ade0 tx=0x7f50400319f0 comp rx=0 tx=0).stop 2024-09-19T07:32:35.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.150+0000 7f5046800640 1 -- 172.21.15.159:0/2221217455 reap_dead start 2024-09-19T07:32:35.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.150+0000 7f5047200640 1 -- 172.21.15.159:0/2221217455 shutdown_connections 2024-09-19T07:32:35.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.150+0000 7f5047200640 1 -- 172.21.15.159:0/2221217455 >> 172.21.15.159:0/2221217455 conn(0x7f50480ff910 msgr2=0x7f5048100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:35.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.150+0000 7f5047200640 1 -- 172.21.15.159:0/2221217455 shutdown_connections 2024-09-19T07:32:35.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.150+0000 7f5047200640 1 -- 172.21.15.159:0/2221217455 wait complete. 2024-09-19T07:32:36.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:35 smithi159 ceph-mon[30472]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T07:32:36.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:35 smithi203 ceph-mon[35335]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T07:32:36.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:35 smithi189 ceph-mon[34851]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T07:32:36.701 INFO:teuthology.orchestra.run.smithi159.stdout:34359738462 2024-09-19T07:32:36.704 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T07:32:37.074 INFO:teuthology.orchestra.run.smithi159.stdout:201863462971 2024-09-19T07:32:37.074 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T07:32:37.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.379+0000 7ffa65000640 1 -- 172.21.15.159:0/3715574779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa601010d0 msgr2=0x7ffa6010dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:37.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.379+0000 7ffa65000640 1 --2- 172.21.15.159:0/3715574779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa601010d0 0x7ffa6010dd90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009900 tx=0x7ffa4c02a3f0 comp rx=0 tx=0).stop 2024-09-19T07:32:37.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.379+0000 7ffa65000640 1 -- 172.21.15.159:0/3715574779 shutdown_connections 2024-09-19T07:32:37.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.379+0000 7ffa65000640 1 --2- 172.21.15.159:0/3715574779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa6010e2d0 0x7ffa601106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:37.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.379+0000 7ffa65000640 1 --2- 172.21.15.159:0/3715574779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa601010d0 0x7ffa6010dd90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:37.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.379+0000 7ffa65000640 1 --2- 172.21.15.159:0/3715574779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa60100720 0x7ffa60100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:37.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.379+0000 7ffa65000640 1 -- 172.21.15.159:0/3715574779 >> 172.21.15.159:0/3715574779 conn(0x7ffa600fc610 msgr2=0x7ffa600fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:37.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.379+0000 7ffa65000640 1 -- 172.21.15.159:0/3715574779 shutdown_connections 2024-09-19T07:32:37.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.380+0000 7ffa65000640 1 -- 172.21.15.159:0/3715574779 wait complete. 2024-09-19T07:32:37.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.381+0000 7ffa65000640 1 Processor -- start 2024-09-19T07:32:37.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.381+0000 7ffa65000640 1 -- start start 2024-09-19T07:32:37.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.381+0000 7ffa65000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa60100720 0x7ffa60107350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:37.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.381+0000 7ffa65000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa601010d0 0x7ffa60107890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:37.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.381+0000 7ffa65000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa6010e2d0 0x7ffa60102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:37.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.382+0000 7ffa5fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa6010e2d0 0x7ffa60102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:37.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.382+0000 7ffa65000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffa60113fd0 con 0x7ffa6010e2d0 2024-09-19T07:32:37.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.382+0000 7ffa65000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffa60113e50 con 0x7ffa60100720 2024-09-19T07:32:37.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.382+0000 7ffa65000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ffa60114150 con 0x7ffa601010d0 2024-09-19T07:32:37.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.382+0000 7ffa5fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa6010e2d0 0x7ffa60102930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37256/0 (socket says 172.21.15.159:37256) 2024-09-19T07:32:37.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.382+0000 7ffa5fe00640 1 -- 172.21.15.159:0/890459736 learned_addr learned my addr 172.21.15.159:0/890459736 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:37.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.382+0000 7ffa5f400640 1 --2- 172.21.15.159:0/890459736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa60100720 0x7ffa60107350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:37.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.382+0000 7ffa5ea00640 1 --2- 172.21.15.159:0/890459736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa601010d0 0x7ffa60107890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:37.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.383+0000 7ffa5f400640 1 -- 172.21.15.159:0/890459736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa601010d0 msgr2=0x7ffa60107890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:37.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.383+0000 7ffa5f400640 1 --2- 172.21.15.159:0/890459736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa601010d0 0x7ffa60107890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:37.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.383+0000 7ffa5f400640 1 -- 172.21.15.159:0/890459736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa6010e2d0 msgr2=0x7ffa60102930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:37.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.383+0000 7ffa5f400640 1 --2- 172.21.15.159:0/890459736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa6010e2d0 0x7ffa60102930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:37.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.383+0000 7ffa5f400640 1 -- 172.21.15.159:0/890459736 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa601030b0 con 0x7ffa60100720 2024-09-19T07:32:37.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.383+0000 7ffa5f400640 1 --2- 172.21.15.159:0/890459736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa60100720 0x7ffa60107350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000e9f0 tx=0x7ffa5000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:37.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.384+0000 7ffa57e00640 1 -- 172.21.15.159:0/890459736 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5000ccc0 con 0x7ffa60100720 2024-09-19T07:32:37.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.384+0000 7ffa65000640 1 -- 172.21.15.159:0/890459736 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa601033a0 con 0x7ffa60100720 2024-09-19T07:32:37.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.384+0000 7ffa57e00640 1 -- 172.21.15.159:0/890459736 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa50010910 con 0x7ffa60100720 2024-09-19T07:32:37.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.384+0000 7ffa65000640 1 -- 172.21.15.159:0/890459736 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa6010d270 con 0x7ffa60100720 2024-09-19T07:32:37.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.385+0000 7ffa57e00640 1 -- 172.21.15.159:0/890459736 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa500189c0 con 0x7ffa60100720 2024-09-19T07:32:37.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.385+0000 7ffa65000640 1 -- 172.21.15.159:0/890459736 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa40001030 con 0x7ffa60100720 2024-09-19T07:32:37.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.389+0000 7ffa57e00640 1 -- 172.21.15.159:0/890459736 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ffa50018b60 con 0x7ffa60100720 2024-09-19T07:32:37.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.390+0000 7ffa57e00640 1 --2- 172.21.15.159:0/890459736 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ffa440a3350 0x7ffa440a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:37.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.390+0000 7ffa57e00640 1 -- 172.21.15.159:0/890459736 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ffa500d03e0 con 0x7ffa60100720 2024-09-19T07:32:37.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.390+0000 7ffa57e00640 1 --2- 172.21.15.159:0/890459736 >> [v2:172.21.15.203:6808/4212601512,v1:172.21.15.203:6809/4212601512] conn(0x7ffa440ae6a0 0x7ffa440b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:37.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.390+0000 7ffa5ea00640 1 --2- 172.21.15.159:0/890459736 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ffa440a3350 0x7ffa440a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:37.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.390+0000 7ffa57e00640 1 -- 172.21.15.159:0/890459736 --> [v2:172.21.15.203:6808/4212601512,v1:172.21.15.203:6809/4212601512] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa50024e70 con 0x7ffa440ae6a0 2024-09-19T07:32:37.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.390+0000 7ffa57e00640 1 -- 172.21.15.159:0/890459736 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7ffa500d0830 con 0x7ffa60100720 2024-09-19T07:32:37.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.391+0000 7ffa5fe00640 1 --2- 172.21.15.159:0/890459736 >> [v2:172.21.15.203:6808/4212601512,v1:172.21.15.203:6809/4212601512] conn(0x7ffa440ae6a0 0x7ffa440b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:37.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.391+0000 7ffa5fe00640 1 --2- 172.21.15.159:0/890459736 >> [v2:172.21.15.203:6808/4212601512,v1:172.21.15.203:6809/4212601512] conn(0x7ffa440ae6a0 0x7ffa440b0b40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:37.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.392+0000 7ffa5ea00640 1 --2- 172.21.15.159:0/890459736 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ffa440a3350 0x7ffa440a5810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa6010e910 tx=0x7ffa4c005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:37.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.392+0000 7ffa57e00640 1 -- 172.21.15.159:0/890459736 <== osd.5 v2:172.21.15.203:6808/4212601512 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa50024e70 con 0x7ffa440ae6a0 2024-09-19T07:32:37.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.407+0000 7ffa65000640 1 -- 172.21.15.159:0/890459736 --> [v2:172.21.15.203:6808/4212601512,v1:172.21.15.203:6809/4212601512] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa40002dd0 con 0x7ffa440ae6a0 2024-09-19T07:32:37.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.408+0000 7ffa57e00640 1 -- 172.21.15.159:0/890459736 <== osd.5 v2:172.21.15.203:6808/4212601512 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffa40002dd0 con 0x7ffa440ae6a0 2024-09-19T07:32:37.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.408+0000 7ffa65000640 1 -- 172.21.15.159:0/890459736 >> [v2:172.21.15.203:6808/4212601512,v1:172.21.15.203:6809/4212601512] conn(0x7ffa440ae6a0 msgr2=0x7ffa440b0b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:37.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.408+0000 7ffa65000640 1 --2- 172.21.15.159:0/890459736 >> [v2:172.21.15.203:6808/4212601512,v1:172.21.15.203:6809/4212601512] conn(0x7ffa440ae6a0 0x7ffa440b0b40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:37.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.409+0000 7ffa65000640 1 -- 172.21.15.159:0/890459736 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ffa440a3350 msgr2=0x7ffa440a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:37.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.409+0000 7ffa65000640 1 --2- 172.21.15.159:0/890459736 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ffa440a3350 0x7ffa440a5810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa6010e910 tx=0x7ffa4c005a20 comp rx=0 tx=0).stop 2024-09-19T07:32:37.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.409+0000 7ffa65000640 1 -- 172.21.15.159:0/890459736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa60100720 msgr2=0x7ffa60107350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:37.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.409+0000 7ffa65000640 1 --2- 172.21.15.159:0/890459736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa60100720 0x7ffa60107350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000e9f0 tx=0x7ffa5000eeb0 comp rx=0 tx=0).stop 2024-09-19T07:32:37.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.409+0000 7ffa5fe00640 1 -- 172.21.15.159:0/890459736 reap_dead start 2024-09-19T07:32:37.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.409+0000 7ffa65000640 1 -- 172.21.15.159:0/890459736 shutdown_connections 2024-09-19T07:32:37.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.409+0000 7ffa65000640 1 -- 172.21.15.159:0/890459736 >> 172.21.15.159:0/890459736 conn(0x7ffa600fc610 msgr2=0x7ffa600fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:37.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.409+0000 7ffa65000640 1 -- 172.21.15.159:0/890459736 shutdown_connections 2024-09-19T07:32:37.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:35.410+0000 7ffa65000640 1 -- 172.21.15.159:0/890459736 wait complete. 2024-09-19T07:32:38.034 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:37 smithi159 ceph-mon[30472]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T07:32:38.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:37 smithi203 ceph-mon[35335]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T07:32:38.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:37 smithi189 ceph-mon[34851]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T07:32:38.542 INFO:teuthology.orchestra.run.smithi159.stdout:167503724609 2024-09-19T07:32:38.543 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T07:32:38.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.837+0000 7fb60e400640 1 -- 172.21.15.159:0/1327414742 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb608102340 msgr2=0x7fb60810e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:38.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.837+0000 7fb60e400640 1 --2- 172.21.15.159:0/1327414742 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb608102340 0x7fb60810e870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb600009cd0 tx=0x7fb60002a8e0 comp rx=0 tx=0).stop 2024-09-19T07:32:38.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.837+0000 7fb60e400640 1 -- 172.21.15.159:0/1327414742 shutdown_connections 2024-09-19T07:32:38.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.837+0000 7fb60e400640 1 --2- 172.21.15.159:0/1327414742 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb608102340 0x7fb60810e870 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:38.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.837+0000 7fb60e400640 1 --2- 172.21.15.159:0/1327414742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6081019a0 0x7fb608101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:38.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.837+0000 7fb60e400640 1 --2- 172.21.15.159:0/1327414742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6081079a0 0x7fb608107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:38.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.837+0000 7fb60e400640 1 -- 172.21.15.159:0/1327414742 >> 172.21.15.159:0/1327414742 conn(0x7fb6080fd700 msgr2=0x7fb6080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:38.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.837+0000 7fb60e400640 1 -- 172.21.15.159:0/1327414742 shutdown_connections 2024-09-19T07:32:38.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.838+0000 7fb60e400640 1 -- 172.21.15.159:0/1327414742 wait complete. 2024-09-19T07:32:38.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.838+0000 7fb60e400640 1 Processor -- start 2024-09-19T07:32:38.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.839+0000 7fb60e400640 1 -- start start 2024-09-19T07:32:38.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.839+0000 7fb60e400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6081019a0 0x7fb60819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:38.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.839+0000 7fb60e400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb608102340 0x7fb60819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:38.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.839+0000 7fb60e400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6081079a0 0x7fb60819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:38.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.839+0000 7fb60e400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb608111da0 con 0x7fb6081019a0 2024-09-19T07:32:38.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.839+0000 7fb60e400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb608111c20 con 0x7fb608102340 2024-09-19T07:32:38.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.839+0000 7fb60e400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb608111f20 con 0x7fb6081079a0 2024-09-19T07:32:38.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.839+0000 7fb60d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6081019a0 0x7fb60819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:38.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.839+0000 7fb60d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6081019a0 0x7fb60819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37278/0 (socket says 172.21.15.159:37278) 2024-09-19T07:32:38.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.839+0000 7fb60d000640 1 -- 172.21.15.159:0/2800611303 learned_addr learned my addr 172.21.15.159:0/2800611303 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:38.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.839+0000 7fb607e00640 1 --2- 172.21.15.159:0/2800611303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb608102340 0x7fb60819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:38.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.840+0000 7fb60da00640 1 --2- 172.21.15.159:0/2800611303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6081079a0 0x7fb60819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:38.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.840+0000 7fb60d000640 1 -- 172.21.15.159:0/2800611303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6081079a0 msgr2=0x7fb60819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:38.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.840+0000 7fb60d000640 1 --2- 172.21.15.159:0/2800611303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6081079a0 0x7fb60819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:38.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.840+0000 7fb60d000640 1 -- 172.21.15.159:0/2800611303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb608102340 msgr2=0x7fb60819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:38.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.840+0000 7fb60d000640 1 --2- 172.21.15.159:0/2800611303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb608102340 0x7fb60819a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:38.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.840+0000 7fb60d000640 1 -- 172.21.15.159:0/2800611303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60819f3e0 con 0x7fb6081019a0 2024-09-19T07:32:38.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.840+0000 7fb60d000640 1 --2- 172.21.15.159:0/2800611303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6081019a0 0x7fb60819a430 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00e9f0 tx=0x7fb5fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:38.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.840+0000 7fb605600640 1 -- 172.21.15.159:0/2800611303 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc00cd80 con 0x7fb6081019a0 2024-09-19T07:32:38.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.840+0000 7fb605600640 1 -- 172.21.15.159:0/2800611303 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5fc010910 con 0x7fb6081019a0 2024-09-19T07:32:38.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.840+0000 7fb60e400640 1 -- 172.21.15.159:0/2800611303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60819f6d0 con 0x7fb6081019a0 2024-09-19T07:32:38.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.840+0000 7fb605600640 1 -- 172.21.15.159:0/2800611303 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc018990 con 0x7fb6081019a0 2024-09-19T07:32:38.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.840+0000 7fb60e400640 1 -- 172.21.15.159:0/2800611303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6081a6fb0 con 0x7fb6081019a0 2024-09-19T07:32:38.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.841+0000 7fb60e400640 1 -- 172.21.15.159:0/2800611303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb6080fe7f0 con 0x7fb6081019a0 2024-09-19T07:32:38.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.843+0000 7fb605600640 1 -- 172.21.15.159:0/2800611303 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb5fc010ac0 con 0x7fb6081019a0 2024-09-19T07:32:38.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.844+0000 7fb605600640 1 --2- 172.21.15.159:0/2800611303 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb5f00a32d0 0x7fb5f00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:38.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.845+0000 7fb605600640 1 -- 172.21.15.159:0/2800611303 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fb5fc0d0870 con 0x7fb6081019a0 2024-09-19T07:32:38.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.845+0000 7fb607e00640 1 --2- 172.21.15.159:0/2800611303 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb5f00a32d0 0x7fb5f00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:38.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.845+0000 7fb605600640 1 --2- 172.21.15.159:0/2800611303 >> [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380] conn(0x7fb5f00ae620 0x7fb5f00b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:38.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.845+0000 7fb605600640 1 -- 172.21.15.159:0/2800611303 --> [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb5fc010760 con 0x7fb5f00ae620 2024-09-19T07:32:38.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.845+0000 7fb605600640 1 -- 172.21.15.159:0/2800611303 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fb5fc0d0c20 con 0x7fb6081019a0 2024-09-19T07:32:38.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.845+0000 7fb60da00640 1 --2- 172.21.15.159:0/2800611303 >> [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380] conn(0x7fb5f00ae620 0x7fb5f00b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:38.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.845+0000 7fb607e00640 1 --2- 172.21.15.159:0/2800611303 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb5f00a32d0 0x7fb5f00a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb60810ed20 tx=0x7fb5f400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:38.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.846+0000 7fb60da00640 1 --2- 172.21.15.159:0/2800611303 >> [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380] conn(0x7fb5f00ae620 0x7fb5f00b0ac0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:38.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.847+0000 7fb605600640 1 -- 172.21.15.159:0/2800611303 <== osd.3 v2:172.21.15.159:6810/109457380 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5fc010760 con 0x7fb5f00ae620 2024-09-19T07:32:38.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.862+0000 7fb60e400640 1 -- 172.21.15.159:0/2800611303 --> [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb6081030e0 con 0x7fb5f00ae620 2024-09-19T07:32:38.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.863+0000 7fb605600640 1 -- 172.21.15.159:0/2800611303 <== osd.3 v2:172.21.15.159:6810/109457380 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb6081030e0 con 0x7fb5f00ae620 2024-09-19T07:32:38.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.863+0000 7fb60e400640 1 -- 172.21.15.159:0/2800611303 >> [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380] conn(0x7fb5f00ae620 msgr2=0x7fb5f00b0ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:38.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.863+0000 7fb60e400640 1 --2- 172.21.15.159:0/2800611303 >> [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380] conn(0x7fb5f00ae620 0x7fb5f00b0ac0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:38.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.863+0000 7fb60e400640 1 -- 172.21.15.159:0/2800611303 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb5f00a32d0 msgr2=0x7fb5f00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:38.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.863+0000 7fb60e400640 1 --2- 172.21.15.159:0/2800611303 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb5f00a32d0 0x7fb5f00a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb60810ed20 tx=0x7fb5f400b410 comp rx=0 tx=0).stop 2024-09-19T07:32:38.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.863+0000 7fb60e400640 1 -- 172.21.15.159:0/2800611303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6081019a0 msgr2=0x7fb60819a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:38.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.863+0000 7fb60e400640 1 --2- 172.21.15.159:0/2800611303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6081019a0 0x7fb60819a430 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00e9f0 tx=0x7fb5fc00eeb0 comp rx=0 tx=0).stop 2024-09-19T07:32:38.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.863+0000 7fb60da00640 1 -- 172.21.15.159:0/2800611303 reap_dead start 2024-09-19T07:32:38.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.863+0000 7fb60e400640 1 -- 172.21.15.159:0/2800611303 shutdown_connections 2024-09-19T07:32:38.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.863+0000 7fb60e400640 1 -- 172.21.15.159:0/2800611303 >> 172.21.15.159:0/2800611303 conn(0x7fb6080fd700 msgr2=0x7fb6080fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:38.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.864+0000 7fb60e400640 1 -- 172.21.15.159:0/2800611303 shutdown_connections 2024-09-19T07:32:38.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:38.864+0000 7fb60e400640 1 -- 172.21.15.159:0/2800611303 wait complete. 2024-09-19T07:32:39.501 INFO:teuthology.orchestra.run.smithi159.stdout:296352743468 2024-09-19T07:32:39.502 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-19T07:32:39.693 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:40.112 INFO:teuthology.orchestra.run.smithi159.stdout:111669149773 2024-09-19T07:32:40.112 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T07:32:40.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:39 smithi159 ceph-mon[30472]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T07:32:40.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:39 smithi203 ceph-mon[35335]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T07:32:40.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:39 smithi189 ceph-mon[34851]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T07:32:40.457 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:40.469 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:40.470 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:40.473 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:40.481 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:40.482 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:40.487 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:40.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.731+0000 7f9947200640 1 -- 172.21.15.159:0/3160368935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9948100720 msgr2=0x7f9948100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:40.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.731+0000 7f9947200640 1 --2- 172.21.15.159:0/3160368935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9948100720 0x7f9948100b00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9938009900 tx=0x7f993802a410 comp rx=0 tx=0).stop 2024-09-19T07:32:40.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.731+0000 7f9947200640 1 -- 172.21.15.159:0/3160368935 shutdown_connections 2024-09-19T07:32:40.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.731+0000 7f9947200640 1 --2- 172.21.15.159:0/3160368935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f994810e2d0 0x7f99481106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:40.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.731+0000 7f9947200640 1 --2- 172.21.15.159:0/3160368935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99481010d0 0x7f994810dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:40.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.731+0000 7f9947200640 1 --2- 172.21.15.159:0/3160368935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9948100720 0x7f9948100b00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:40.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.731+0000 7f9947200640 1 -- 172.21.15.159:0/3160368935 >> 172.21.15.159:0/3160368935 conn(0x7f99480fc610 msgr2=0x7f99480fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:40.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.731+0000 7f9947200640 1 -- 172.21.15.159:0/3160368935 shutdown_connections 2024-09-19T07:32:40.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.731+0000 7f9947200640 1 -- 172.21.15.159:0/3160368935 wait complete. 2024-09-19T07:32:40.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.732+0000 7f9947200640 1 Processor -- start 2024-09-19T07:32:40.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.732+0000 7f9947200640 1 -- start start 2024-09-19T07:32:40.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.733+0000 7f9947200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9948100720 0x7f994819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:40.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.733+0000 7f9947200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99481010d0 0x7f994819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:40.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.733+0000 7f9947200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f994810e2d0 0x7f994819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:40.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.733+0000 7f9947200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9948114040 con 0x7f994810e2d0 2024-09-19T07:32:40.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.733+0000 7f9947200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9948113ec0 con 0x7f99481010d0 2024-09-19T07:32:40.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.733+0000 7f9947200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f99481141c0 con 0x7f9948100720 2024-09-19T07:32:40.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.733+0000 7f9945400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99481010d0 0x7f994819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:40.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.733+0000 7f9946800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f994810e2d0 0x7f994819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:40.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.733+0000 7f9945400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99481010d0 0x7f994819aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:39604/0 (socket says 172.21.15.159:39604) 2024-09-19T07:32:40.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.733+0000 7f9946800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f994810e2d0 0x7f994819ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37302/0 (socket says 172.21.15.159:37302) 2024-09-19T07:32:40.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.733+0000 7f9945e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9948100720 0x7f994819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:40.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.733+0000 7f9945400640 1 -- 172.21.15.159:0/159322849 learned_addr learned my addr 172.21.15.159:0/159322849 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:40.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.734+0000 7f9946800640 1 -- 172.21.15.159:0/159322849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9948100720 msgr2=0x7f994819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:40.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.734+0000 7f9946800640 1 --2- 172.21.15.159:0/159322849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9948100720 0x7f994819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:40.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.734+0000 7f9946800640 1 -- 172.21.15.159:0/159322849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99481010d0 msgr2=0x7f994819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:40.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.734+0000 7f9946800640 1 --2- 172.21.15.159:0/159322849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99481010d0 0x7f994819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:40.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.734+0000 7f9946800640 1 -- 172.21.15.159:0/159322849 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994819f510 con 0x7f994810e2d0 2024-09-19T07:32:40.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.734+0000 7f9946800640 1 --2- 172.21.15.159:0/159322849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f994810e2d0 0x7f994819ee30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9948102900 tx=0x7f994000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:40.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.734+0000 7f993ea00640 1 -- 172.21.15.159:0/159322849 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994000cdf0 con 0x7f994810e2d0 2024-09-19T07:32:40.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.734+0000 7f993ea00640 1 -- 172.21.15.159:0/159322849 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9940005020 con 0x7f994810e2d0 2024-09-19T07:32:40.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.734+0000 7f993ea00640 1 -- 172.21.15.159:0/159322849 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940010670 con 0x7f994810e2d0 2024-09-19T07:32:40.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.734+0000 7f9947200640 1 -- 172.21.15.159:0/159322849 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994819f800 con 0x7f994810e2d0 2024-09-19T07:32:40.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.735+0000 7f9947200640 1 -- 172.21.15.159:0/159322849 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99481a70e0 con 0x7f994810e2d0 2024-09-19T07:32:40.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.736+0000 7f9947200640 1 -- 172.21.15.159:0/159322849 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9948062900 con 0x7f994810e2d0 2024-09-19T07:32:40.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.737+0000 7f993ea00640 1 -- 172.21.15.159:0/159322849 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f99400054d0 con 0x7f994810e2d0 2024-09-19T07:32:40.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.739+0000 7f993ea00640 1 --2- 172.21.15.159:0/159322849 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f992c0a3350 0x7f992c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:40.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.739+0000 7f993ea00640 1 -- 172.21.15.159:0/159322849 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f99400d0070 con 0x7f994810e2d0 2024-09-19T07:32:40.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.739+0000 7f9945e00640 1 --2- 172.21.15.159:0/159322849 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f992c0a3350 0x7f992c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:40.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.739+0000 7f993ea00640 1 --2- 172.21.15.159:0/159322849 >> [v2:172.21.15.203:6816/4062926645,v1:172.21.15.203:6817/4062926645] conn(0x7f992c0ae6a0 0x7f992c0b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:40.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.739+0000 7f993ea00640 1 -- 172.21.15.159:0/159322849 --> [v2:172.21.15.203:6816/4062926645,v1:172.21.15.203:6817/4062926645] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9940017ec0 con 0x7f992c0ae6a0 2024-09-19T07:32:40.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.739+0000 7f993ea00640 1 -- 172.21.15.159:0/159322849 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f99400d7050 con 0x7f994810e2d0 2024-09-19T07:32:40.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.740+0000 7f9945400640 1 --2- 172.21.15.159:0/159322849 >> [v2:172.21.15.203:6816/4062926645,v1:172.21.15.203:6817/4062926645] conn(0x7f992c0ae6a0 0x7f992c0b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:40.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.740+0000 7f9945e00640 1 --2- 172.21.15.159:0/159322849 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f992c0a3350 0x7f992c0a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9938000c00 tx=0x7f9938005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:40.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.740+0000 7f9945400640 1 --2- 172.21.15.159:0/159322849 >> [v2:172.21.15.203:6816/4062926645,v1:172.21.15.203:6817/4062926645] conn(0x7f992c0ae6a0 0x7f992c0b0b40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:40.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.741+0000 7f993ea00640 1 -- 172.21.15.159:0/159322849 <== osd.8 v2:172.21.15.203:6816/4062926645 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9940017ec0 con 0x7f992c0ae6a0 2024-09-19T07:32:40.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.760+0000 7f9947200640 1 -- 172.21.15.159:0/159322849 --> [v2:172.21.15.203:6816/4062926645,v1:172.21.15.203:6817/4062926645] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9948062d80 con 0x7f992c0ae6a0 2024-09-19T07:32:40.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.761+0000 7f993ea00640 1 -- 172.21.15.159:0/159322849 <== osd.8 v2:172.21.15.203:6816/4062926645 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9948062d80 con 0x7f992c0ae6a0 2024-09-19T07:32:40.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.761+0000 7f9947200640 1 -- 172.21.15.159:0/159322849 >> [v2:172.21.15.203:6816/4062926645,v1:172.21.15.203:6817/4062926645] conn(0x7f992c0ae6a0 msgr2=0x7f992c0b0b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:40.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.761+0000 7f9947200640 1 --2- 172.21.15.159:0/159322849 >> [v2:172.21.15.203:6816/4062926645,v1:172.21.15.203:6817/4062926645] conn(0x7f992c0ae6a0 0x7f992c0b0b40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:40.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.762+0000 7f9947200640 1 -- 172.21.15.159:0/159322849 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f992c0a3350 msgr2=0x7f992c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:40.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.762+0000 7f9947200640 1 --2- 172.21.15.159:0/159322849 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f992c0a3350 0x7f992c0a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9938000c00 tx=0x7f9938005b50 comp rx=0 tx=0).stop 2024-09-19T07:32:40.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.762+0000 7f9947200640 1 -- 172.21.15.159:0/159322849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f994810e2d0 msgr2=0x7f994819ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:40.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.762+0000 7f9947200640 1 --2- 172.21.15.159:0/159322849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f994810e2d0 0x7f994819ee30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9948102900 tx=0x7f994000ef10 comp rx=0 tx=0).stop 2024-09-19T07:32:40.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.762+0000 7f9946800640 1 -- 172.21.15.159:0/159322849 reap_dead start 2024-09-19T07:32:40.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.762+0000 7f9947200640 1 -- 172.21.15.159:0/159322849 shutdown_connections 2024-09-19T07:32:40.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.762+0000 7f9947200640 1 -- 172.21.15.159:0/159322849 >> 172.21.15.159:0/159322849 conn(0x7f99480fc610 msgr2=0x7f99480fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:40.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.762+0000 7f9947200640 1 -- 172.21.15.159:0/159322849 shutdown_connections 2024-09-19T07:32:40.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:40.762+0000 7f9947200640 1 -- 172.21.15.159:0/159322849 wait complete. 2024-09-19T07:32:41.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:41 smithi159 ceph-mon[30472]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T07:32:42.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:41 smithi203 ceph-mon[35335]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T07:32:42.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:41 smithi189 ceph-mon[34851]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T07:32:42.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.020+0000 7f12a3800640 1 -- 172.21.15.159:0/3389707167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f129c109120 msgr2=0x7f129c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:42.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.020+0000 7f12a3800640 1 --2- 172.21.15.159:0/3389707167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f129c109120 0x7f129c110ca0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1294009ca0 tx=0x7f129402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:32:42.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.020+0000 7f12a3800640 1 -- 172.21.15.159:0/3389707167 shutdown_connections 2024-09-19T07:32:42.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.020+0000 7f12a3800640 1 --2- 172.21.15.159:0/3389707167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f129c109120 0x7f129c110ca0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:42.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.020+0000 7f12a3800640 1 --2- 172.21.15.159:0/3389707167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c1049a0 0x7f129c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:42.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.020+0000 7f12a3800640 1 --2- 172.21.15.159:0/3389707167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f129c103ff0 0x7f129c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:42.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.020+0000 7f12a3800640 1 -- 172.21.15.159:0/3389707167 >> 172.21.15.159:0/3389707167 conn(0x7f129c0ffa90 msgr2=0x7f129c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:42.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.020+0000 7f12a3800640 1 -- 172.21.15.159:0/3389707167 shutdown_connections 2024-09-19T07:32:42.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.020+0000 7f12a3800640 1 -- 172.21.15.159:0/3389707167 wait complete. 2024-09-19T07:32:42.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.021+0000 7f12a3800640 1 Processor -- start 2024-09-19T07:32:42.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.021+0000 7f12a3800640 1 -- start start 2024-09-19T07:32:42.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.022+0000 7f12a3800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f129c103ff0 0x7f129c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:42.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.022+0000 7f12a3800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c1049a0 0x7f129c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:42.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.022+0000 7f12a3800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f129c109120 0x7f129c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:42.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.022+0000 7f12a3800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1141a0 con 0x7f129c109120 2024-09-19T07:32:42.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.022+0000 7f12a3800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f129c114020 con 0x7f129c103ff0 2024-09-19T07:32:42.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.022+0000 7f12a3800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f129c114320 con 0x7f129c1049a0 2024-09-19T07:32:42.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.022+0000 7f12a1800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f129c109120 0x7f129c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:42.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.022+0000 7f12a1800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f129c109120 0x7f129c19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39482/0 (socket says 172.21.15.159:39482) 2024-09-19T07:32:42.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.022+0000 7f12a1800640 1 -- 172.21.15.159:0/3867687495 learned_addr learned my addr 172.21.15.159:0/3867687495 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:42.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.022+0000 7f129be00640 1 --2- 172.21.15.159:0/3867687495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c1049a0 0x7f129c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:42.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.022+0000 7f12a0e00640 1 --2- 172.21.15.159:0/3867687495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f129c103ff0 0x7f129c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:42.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.022+0000 7f12a1800640 1 -- 172.21.15.159:0/3867687495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c1049a0 msgr2=0x7f129c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:42.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.022+0000 7f12a1800640 1 --2- 172.21.15.159:0/3867687495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c1049a0 0x7f129c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:42.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.022+0000 7f12a1800640 1 -- 172.21.15.159:0/3867687495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f129c103ff0 msgr2=0x7f129c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:42.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.022+0000 7f12a1800640 1 --2- 172.21.15.159:0/3867687495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f129c103ff0 0x7f129c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:42.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.022+0000 7f12a1800640 1 -- 172.21.15.159:0/3867687495 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129c19f5b0 con 0x7f129c109120 2024-09-19T07:32:42.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.023+0000 7f12a1800640 1 --2- 172.21.15.159:0/3867687495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f129c109120 0x7f129c19eed0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1294005e30 tx=0x7f1294003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:42.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.023+0000 7f1299600640 1 -- 172.21.15.159:0/3867687495 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294031650 con 0x7f129c109120 2024-09-19T07:32:42.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.023+0000 7f1299600640 1 -- 172.21.15.159:0/3867687495 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12940317f0 con 0x7f129c109120 2024-09-19T07:32:42.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.023+0000 7f12a3800640 1 -- 172.21.15.159:0/3867687495 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129c19f840 con 0x7f129c109120 2024-09-19T07:32:42.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.023+0000 7f1299600640 1 -- 172.21.15.159:0/3867687495 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129402f9d0 con 0x7f129c109120 2024-09-19T07:32:42.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.023+0000 7f12a3800640 1 -- 172.21.15.159:0/3867687495 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f129c1a7180 con 0x7f129c109120 2024-09-19T07:32:42.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.024+0000 7f12a3800640 1 -- 172.21.15.159:0/3867687495 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129c1043d0 con 0x7f129c109120 2024-09-19T07:32:42.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.026+0000 7f1299600640 1 -- 172.21.15.159:0/3867687495 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f129402fbf0 con 0x7f129c109120 2024-09-19T07:32:42.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.027+0000 7f1299600640 1 --2- 172.21.15.159:0/3867687495 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f12840a32d0 0x7f12840a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:42.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.028+0000 7f1299600640 1 -- 172.21.15.159:0/3867687495 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f12940f10c0 con 0x7f129c109120 2024-09-19T07:32:42.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.028+0000 7f12a0e00640 1 --2- 172.21.15.159:0/3867687495 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f12840a32d0 0x7f12840a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:42.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.028+0000 7f12a0e00640 1 --2- 172.21.15.159:0/3867687495 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f12840a32d0 0x7f12840a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f12900095f0 tx=0x7f1290005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:42.334 INFO:teuthology.orchestra.run.smithi159.stdout:34359738463 2024-09-19T07:32:42.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.032+0000 7f1299600640 1 -- 172.21.15.159:0/3867687495 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f12940bd980 con 0x7f129c109120 2024-09-19T07:32:42.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.216+0000 7f12a3800640 1 -- 172.21.15.159:0/3867687495 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f129c1a73b0 con 0x7f129c109120 2024-09-19T07:32:42.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.217+0000 7f1299600640 1 -- 172.21.15.159:0/3867687495 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f12940bd280 con 0x7f129c109120 2024-09-19T07:32:42.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.219+0000 7f12a3800640 1 -- 172.21.15.159:0/3867687495 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f12840a32d0 msgr2=0x7f12840a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:42.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.219+0000 7f12a3800640 1 --2- 172.21.15.159:0/3867687495 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f12840a32d0 0x7f12840a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f12900095f0 tx=0x7f1290005f90 comp rx=0 tx=0).stop 2024-09-19T07:32:42.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.219+0000 7f12a3800640 1 -- 172.21.15.159:0/3867687495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f129c109120 msgr2=0x7f129c19eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:42.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.219+0000 7f12a3800640 1 --2- 172.21.15.159:0/3867687495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f129c109120 0x7f129c19eed0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1294005e30 tx=0x7f1294003d00 comp rx=0 tx=0).stop 2024-09-19T07:32:42.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.219+0000 7f12a3800640 1 -- 172.21.15.159:0/3867687495 shutdown_connections 2024-09-19T07:32:42.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.219+0000 7f12a3800640 1 --2- 172.21.15.159:0/3867687495 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f12840a32d0 0x7f12840a5790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:42.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.219+0000 7f12a3800640 1 --2- 172.21.15.159:0/3867687495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f129c109120 0x7f129c19eed0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:42.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.219+0000 7f12a3800640 1 --2- 172.21.15.159:0/3867687495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c1049a0 0x7f129c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:42.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.219+0000 7f12a3800640 1 --2- 172.21.15.159:0/3867687495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f129c103ff0 0x7f129c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:42.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.219+0000 7f12a3800640 1 -- 172.21.15.159:0/3867687495 >> 172.21.15.159:0/3867687495 conn(0x7f129c0ffa90 msgr2=0x7f129c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:42.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.219+0000 7f12a3800640 1 -- 172.21.15.159:0/3867687495 shutdown_connections 2024-09-19T07:32:42.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.219+0000 7f12a3800640 1 -- 172.21.15.159:0/3867687495 wait complete. 2024-09-19T07:32:42.977 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:42 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3867687495' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T07:32:43.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:42 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3867687495' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T07:32:43.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:42 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3867687495' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T07:32:43.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.795+0000 7ff055200640 1 -- 172.21.15.159:0/3927721373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff050074620 msgr2=0x7ff050074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:43.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.795+0000 7ff055200640 1 --2- 172.21.15.159:0/3927721373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff050074620 0x7ff050074a00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff040009900 tx=0x7ff04002a480 comp rx=0 tx=0).stop 2024-09-19T07:32:43.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.795+0000 7ff055200640 1 -- 172.21.15.159:0/3927721373 shutdown_connections 2024-09-19T07:32:43.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.795+0000 7ff055200640 1 --2- 172.21.15.159:0/3927721373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff05010e3c0 0x7ff050110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:43.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.795+0000 7ff055200640 1 --2- 172.21.15.159:0/3927721373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff050074f40 0x7ff05010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:43.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.795+0000 7ff055200640 1 --2- 172.21.15.159:0/3927721373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff050074620 0x7ff050074a00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:43.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.795+0000 7ff055200640 1 -- 172.21.15.159:0/3927721373 >> 172.21.15.159:0/3927721373 conn(0x7ff0500fd8c0 msgr2=0x7ff0500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:43.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.795+0000 7ff055200640 1 -- 172.21.15.159:0/3927721373 shutdown_connections 2024-09-19T07:32:43.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.795+0000 7ff055200640 1 -- 172.21.15.159:0/3927721373 wait complete. 2024-09-19T07:32:43.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.796+0000 7ff055200640 1 Processor -- start 2024-09-19T07:32:43.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.796+0000 7ff055200640 1 -- start start 2024-09-19T07:32:43.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.796+0000 7ff055200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff050074620 0x7ff05019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:43.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.796+0000 7ff055200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff050074f40 0x7ff05019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:43.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.796+0000 7ff055200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff05010e3c0 0x7ff05019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:43.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.796+0000 7ff055200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff050114170 con 0x7ff050074f40 2024-09-19T07:32:43.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.796+0000 7ff055200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff050113ff0 con 0x7ff050074620 2024-09-19T07:32:43.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.796+0000 7ff055200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff0501142f0 con 0x7ff05010e3c0 2024-09-19T07:32:43.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.797+0000 7ff045400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff050074f40 0x7ff05019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:43.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.797+0000 7ff045400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff050074f40 0x7ff05019abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39506/0 (socket says 172.21.15.159:39506) 2024-09-19T07:32:43.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.797+0000 7ff045400640 1 -- 172.21.15.159:0/2585362973 learned_addr learned my addr 172.21.15.159:0/2585362973 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:43.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.797+0000 7ff04de00640 1 --2- 172.21.15.159:0/2585362973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff050074620 0x7ff05019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:43.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.797+0000 7ff04e800640 1 --2- 172.21.15.159:0/2585362973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff05010e3c0 0x7ff05019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:43.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.797+0000 7ff045400640 1 -- 172.21.15.159:0/2585362973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff05010e3c0 msgr2=0x7ff05019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:43.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.797+0000 7ff045400640 1 --2- 172.21.15.159:0/2585362973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff05010e3c0 0x7ff05019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:43.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.797+0000 7ff045400640 1 -- 172.21.15.159:0/2585362973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff050074620 msgr2=0x7ff05019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:43.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.797+0000 7ff045400640 1 --2- 172.21.15.159:0/2585362973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff050074620 0x7ff05019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:43.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.797+0000 7ff045400640 1 -- 172.21.15.159:0/2585362973 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05019f630 con 0x7ff050074f40 2024-09-19T07:32:43.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.797+0000 7ff04de00640 1 --2- 172.21.15.159:0/2585362973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff050074620 0x7ff05019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:32:43.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.798+0000 7ff045400640 1 --2- 172.21.15.159:0/2585362973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff050074f40 0x7ff05019abc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff03800d950 tx=0x7ff03800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:43.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.798+0000 7ff047400640 1 -- 172.21.15.159:0/2585362973 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038017030 con 0x7ff050074f40 2024-09-19T07:32:43.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.798+0000 7ff055200640 1 -- 172.21.15.159:0/2585362973 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05019f920 con 0x7ff050074f40 2024-09-19T07:32:43.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.798+0000 7ff047400640 1 -- 172.21.15.159:0/2585362973 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff038004020 con 0x7ff050074f40 2024-09-19T07:32:43.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.798+0000 7ff047400640 1 -- 172.21.15.159:0/2585362973 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0380047a0 con 0x7ff050074f40 2024-09-19T07:32:43.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.798+0000 7ff055200640 1 -- 172.21.15.159:0/2585362973 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0501a7200 con 0x7ff050074f40 2024-09-19T07:32:43.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.799+0000 7ff055200640 1 -- 172.21.15.159:0/2585362973 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff050074a00 con 0x7ff050074f40 2024-09-19T07:32:43.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.801+0000 7ff047400640 1 -- 172.21.15.159:0/2585362973 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff0380041d0 con 0x7ff050074f40 2024-09-19T07:32:43.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.802+0000 7ff047400640 1 --2- 172.21.15.159:0/2585362973 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff0340a32d0 0x7ff0340a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:43.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.803+0000 7ff047400640 1 -- 172.21.15.159:0/2585362973 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ff0380cf800 con 0x7ff050074f40 2024-09-19T07:32:43.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.803+0000 7ff04de00640 1 --2- 172.21.15.159:0/2585362973 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff0340a32d0 0x7ff0340a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:43.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.803+0000 7ff04de00640 1 --2- 172.21.15.159:0/2585362973 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff0340a32d0 0x7ff0340a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff040009230 tx=0x7ff0400058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:43.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:42.807+0000 7ff047400640 1 -- 172.21.15.159:0/2585362973 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff03809bfb0 con 0x7ff050074f40 2024-09-19T07:32:43.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.000+0000 7ff055200640 1 -- 172.21.15.159:0/2585362973 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff05002d7c0 con 0x7ff050074f40 2024-09-19T07:32:43.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.000+0000 7ff047400640 1 -- 172.21.15.159:0/2585362973 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff03809b8b0 con 0x7ff050074f40 2024-09-19T07:32:43.370 INFO:teuthology.orchestra.run.smithi159.stdout:167503724611 2024-09-19T07:32:43.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.002+0000 7ff055200640 1 -- 172.21.15.159:0/2585362973 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff0340a32d0 msgr2=0x7ff0340a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:43.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.002+0000 7ff055200640 1 --2- 172.21.15.159:0/2585362973 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff0340a32d0 0x7ff0340a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff040009230 tx=0x7ff0400058d0 comp rx=0 tx=0).stop 2024-09-19T07:32:43.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.002+0000 7ff055200640 1 -- 172.21.15.159:0/2585362973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff050074f40 msgr2=0x7ff05019abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:43.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.002+0000 7ff055200640 1 --2- 172.21.15.159:0/2585362973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff050074f40 0x7ff05019abc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff03800d950 tx=0x7ff03800de10 comp rx=0 tx=0).stop 2024-09-19T07:32:43.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.003+0000 7ff055200640 1 -- 172.21.15.159:0/2585362973 shutdown_connections 2024-09-19T07:32:43.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.003+0000 7ff055200640 1 --2- 172.21.15.159:0/2585362973 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff0340a32d0 0x7ff0340a5790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:43.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.003+0000 7ff055200640 1 --2- 172.21.15.159:0/2585362973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff05010e3c0 0x7ff05019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:43.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.003+0000 7ff055200640 1 --2- 172.21.15.159:0/2585362973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff050074f40 0x7ff05019abc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:43.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.003+0000 7ff055200640 1 --2- 172.21.15.159:0/2585362973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff050074620 0x7ff05019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:43.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.003+0000 7ff055200640 1 -- 172.21.15.159:0/2585362973 >> 172.21.15.159:0/2585362973 conn(0x7ff0500fd8c0 msgr2=0x7ff0500ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:43.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.003+0000 7ff055200640 1 -- 172.21.15.159:0/2585362973 shutdown_connections 2024-09-19T07:32:43.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.003+0000 7ff055200640 1 -- 172.21.15.159:0/2585362973 wait complete. 2024-09-19T07:32:44.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.979+0000 7f0aaa400640 1 -- 172.21.15.159:0/2451829708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0aa41034b0 msgr2=0x7f0aa410b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:44.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.979+0000 7f0aaa400640 1 --2- 172.21.15.159:0/2451829708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0aa41034b0 0x7f0aa410b9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009900 tx=0x7f0a9002a480 comp rx=0 tx=0).stop 2024-09-19T07:32:44.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.979+0000 7f0aaa400640 1 -- 172.21.15.159:0/2451829708 shutdown_connections 2024-09-19T07:32:44.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.979+0000 7f0aaa400640 1 --2- 172.21.15.159:0/2451829708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0aa410bfc0 0x7f0aa410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.979+0000 7f0aaa400640 1 --2- 172.21.15.159:0/2451829708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0aa41034b0 0x7f0aa410b9a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.979+0000 7f0aaa400640 1 --2- 172.21.15.159:0/2451829708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa4102b90 0x7f0aa4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.979+0000 7f0aaa400640 1 -- 172.21.15.159:0/2451829708 >> 172.21.15.159:0/2451829708 conn(0x7f0aa40fc610 msgr2=0x7f0aa40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:44.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.979+0000 7f0aaa400640 1 -- 172.21.15.159:0/2451829708 shutdown_connections 2024-09-19T07:32:44.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.979+0000 7f0aaa400640 1 -- 172.21.15.159:0/2451829708 wait complete. 2024-09-19T07:32:44.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.980+0000 7f0aaa400640 1 Processor -- start 2024-09-19T07:32:44.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.980+0000 7f0aaa400640 1 -- start start 2024-09-19T07:32:44.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.980+0000 7f0aaa400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0aa4102b90 0x7f0aa419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:44.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.980+0000 7f0aaa400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0aa41034b0 0x7f0aa419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:44.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.981+0000 7f0aaa400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa410bfc0 0x7f0aa419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:44.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.981+0000 7f0aaa400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa4111dc0 con 0x7f0aa4102b90 2024-09-19T07:32:44.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.981+0000 7f0aaa400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa4111c40 con 0x7f0aa41034b0 2024-09-19T07:32:44.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.981+0000 7f0aaa400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa4111f40 con 0x7f0aa410bfc0 2024-09-19T07:32:44.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.981+0000 7f0aa9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0aa4102b90 0x7f0aa419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:44.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.981+0000 7f0aa9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0aa4102b90 0x7f0aa419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39516/0 (socket says 172.21.15.159:39516) 2024-09-19T07:32:44.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.981+0000 7f0aa9000640 1 -- 172.21.15.159:0/4229433814 learned_addr learned my addr 172.21.15.159:0/4229433814 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:44.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.981+0000 7f0aa3e00640 1 --2- 172.21.15.159:0/4229433814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0aa41034b0 0x7f0aa419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:44.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.981+0000 7f0aa9a00640 1 --2- 172.21.15.159:0/4229433814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa410bfc0 0x7f0aa419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:44.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.981+0000 7f0aa9000640 1 -- 172.21.15.159:0/4229433814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa410bfc0 msgr2=0x7f0aa419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:44.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.981+0000 7f0aa9000640 1 --2- 172.21.15.159:0/4229433814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa410bfc0 0x7f0aa419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.981+0000 7f0aa9000640 1 -- 172.21.15.159:0/4229433814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0aa41034b0 msgr2=0x7f0aa419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:44.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.981+0000 7f0aa9000640 1 --2- 172.21.15.159:0/4229433814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0aa41034b0 0x7f0aa419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.981+0000 7f0aa9000640 1 -- 172.21.15.159:0/4229433814 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa419f470 con 0x7f0aa4102b90 2024-09-19T07:32:44.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.982+0000 7f0aa9000640 1 --2- 172.21.15.159:0/4229433814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0aa4102b90 0x7f0aa419a4c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800e9f0 tx=0x7f0a9800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:44.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.982+0000 7f0aa1600640 1 -- 172.21.15.159:0/4229433814 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9800ccc0 con 0x7f0aa4102b90 2024-09-19T07:32:44.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.982+0000 7f0aa1600640 1 -- 172.21.15.159:0/4229433814 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a98010910 con 0x7f0aa4102b90 2024-09-19T07:32:44.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.982+0000 7f0aa1600640 1 -- 172.21.15.159:0/4229433814 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a980188a0 con 0x7f0aa4102b90 2024-09-19T07:32:44.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.982+0000 7f0aaa400640 1 -- 172.21.15.159:0/4229433814 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa419f760 con 0x7f0aa4102b90 2024-09-19T07:32:44.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.982+0000 7f0aaa400640 1 -- 172.21.15.159:0/4229433814 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa41a7040 con 0x7f0aa4102b90 2024-09-19T07:32:44.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.983+0000 7f0aaa400640 1 -- 172.21.15.159:0/4229433814 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa4102f70 con 0x7f0aa4102b90 2024-09-19T07:32:44.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.985+0000 7f0aa1600640 1 -- 172.21.15.159:0/4229433814 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0a98010ac0 con 0x7f0aa4102b90 2024-09-19T07:32:44.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.986+0000 7f0aa1600640 1 --2- 172.21.15.159:0/4229433814 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0a8c0a32d0 0x7f0a8c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:44.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.986+0000 7f0aa1600640 1 -- 172.21.15.159:0/4229433814 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0a980d02a0 con 0x7f0aa4102b90 2024-09-19T07:32:44.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.987+0000 7f0aa3e00640 1 --2- 172.21.15.159:0/4229433814 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0a8c0a32d0 0x7f0a8c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:44.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.987+0000 7f0aa3e00640 1 --2- 172.21.15.159:0/4229433814 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0a8c0a32d0 0x7f0a8c0a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009230 tx=0x7f0a900058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:44.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:43.990+0000 7f0aa1600640 1 -- 172.21.15.159:0/4229433814 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0a9809ca50 con 0x7f0aa4102b90 2024-09-19T07:32:44.018 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:43 smithi159 ceph-mon[30472]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T07:32:44.018 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:43 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2585362973' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T07:32:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.176+0000 7f0aaa400640 1 -- 172.21.15.159:0/4229433814 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f0aa41a7210 con 0x7f0aa4102b90 2024-09-19T07:32:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.177+0000 7f0aa1600640 1 -- 172.21.15.159:0/4229433814 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f0a9809c350 con 0x7f0aa4102b90 2024-09-19T07:32:44.177 INFO:teuthology.orchestra.run.smithi159.stdout:347892351010 2024-09-19T07:32:44.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.179+0000 7f0aaa400640 1 -- 172.21.15.159:0/4229433814 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0a8c0a32d0 msgr2=0x7f0a8c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:44.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.179+0000 7f0aaa400640 1 --2- 172.21.15.159:0/4229433814 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0a8c0a32d0 0x7f0a8c0a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009230 tx=0x7f0a900058d0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.179+0000 7f0aaa400640 1 -- 172.21.15.159:0/4229433814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0aa4102b90 msgr2=0x7f0aa419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:44.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.179+0000 7f0aaa400640 1 --2- 172.21.15.159:0/4229433814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0aa4102b90 0x7f0aa419a4c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800e9f0 tx=0x7f0a9800edf0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.179+0000 7f0aaa400640 1 -- 172.21.15.159:0/4229433814 shutdown_connections 2024-09-19T07:32:44.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.179+0000 7f0aaa400640 1 --2- 172.21.15.159:0/4229433814 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0a8c0a32d0 0x7f0a8c0a5790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.179+0000 7f0aaa400640 1 --2- 172.21.15.159:0/4229433814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aa410bfc0 0x7f0aa419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.179+0000 7f0aaa400640 1 --2- 172.21.15.159:0/4229433814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0aa41034b0 0x7f0aa419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.179+0000 7f0aaa400640 1 --2- 172.21.15.159:0/4229433814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0aa4102b90 0x7f0aa419a4c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.179+0000 7f0aaa400640 1 -- 172.21.15.159:0/4229433814 >> 172.21.15.159:0/4229433814 conn(0x7f0aa40fc610 msgr2=0x7f0aa410d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:44.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.179+0000 7f0aaa400640 1 -- 172.21.15.159:0/4229433814 shutdown_connections 2024-09-19T07:32:44.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.179+0000 7f0aaa400640 1 -- 172.21.15.159:0/4229433814 wait complete. 2024-09-19T07:32:44.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.236+0000 7f624ac00640 1 -- 172.21.15.159:0/2130867222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6244101dd0 msgr2=0x7f62441021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:44.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.236+0000 7f624ac00640 1 --2- 172.21.15.159:0/2130867222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6244101dd0 0x7f62441021b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6234009900 tx=0x7f623402a3f0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.236+0000 7f624ac00640 1 -- 172.21.15.159:0/2130867222 shutdown_connections 2024-09-19T07:32:44.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.236+0000 7f624ac00640 1 --2- 172.21.15.159:0/2130867222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6244106f00 0x7f624410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.236+0000 7f624ac00640 1 --2- 172.21.15.159:0/2130867222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6244102780 0x7f62441067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.236+0000 7f624ac00640 1 --2- 172.21.15.159:0/2130867222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6244101dd0 0x7f62441021b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.236+0000 7f624ac00640 1 -- 172.21.15.159:0/2130867222 >> 172.21.15.159:0/2130867222 conn(0x7f62440fd8b0 msgr2=0x7f62440ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:44.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.236+0000 7f624ac00640 1 -- 172.21.15.159:0/2130867222 shutdown_connections 2024-09-19T07:32:44.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.236+0000 7f624ac00640 1 -- 172.21.15.159:0/2130867222 wait complete. 2024-09-19T07:32:44.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.238+0000 7f624ac00640 1 Processor -- start 2024-09-19T07:32:44.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.238+0000 7f624ac00640 1 -- start start 2024-09-19T07:32:44.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.238+0000 7f624ac00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6244101dd0 0x7f62440744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:44.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.238+0000 7f624ac00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6244102780 0x7f6244074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:44.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.238+0000 7f624ac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6244106f00 0x7f6244074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:44.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.238+0000 7f624ac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6244111fb0 con 0x7f6244106f00 2024-09-19T07:32:44.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.238+0000 7f624ac00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6244111e30 con 0x7f6244101dd0 2024-09-19T07:32:44.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.238+0000 7f624ac00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6244112130 con 0x7f6244102780 2024-09-19T07:32:44.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.239+0000 7f6248c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6244106f00 0x7f6244074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:44.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.239+0000 7f6243e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6244101dd0 0x7f62440744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:44.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.239+0000 7f6243e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6244101dd0 0x7f62440744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:54070/0 (socket says 172.21.15.159:54070) 2024-09-19T07:32:44.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.239+0000 7f6243400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6244102780 0x7f6244074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:44.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.239+0000 7f6243e00640 1 -- 172.21.15.159:0/3317664972 learned_addr learned my addr 172.21.15.159:0/3317664972 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:44.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.239+0000 7f6243400640 1 -- 172.21.15.159:0/3317664972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6244101dd0 msgr2=0x7f62440744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:44.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.239+0000 7f6243400640 1 --2- 172.21.15.159:0/3317664972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6244101dd0 0x7f62440744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.239+0000 7f6243400640 1 -- 172.21.15.159:0/3317664972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6244106f00 msgr2=0x7f6244074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:44.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.239+0000 7f6243400640 1 --2- 172.21.15.159:0/3317664972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6244106f00 0x7f6244074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.239+0000 7f6243400640 1 -- 172.21.15.159:0/3317664972 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62441a68f0 con 0x7f6244102780 2024-09-19T07:32:44.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.240+0000 7f6243400640 1 --2- 172.21.15.159:0/3317664972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6244102780 0x7f6244074a30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f623000e9f0 tx=0x7f623000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:44.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.240+0000 7f6240c00640 1 -- 172.21.15.159:0/3317664972 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623000ccc0 con 0x7f6244102780 2024-09-19T07:32:44.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.240+0000 7f624ac00640 1 -- 172.21.15.159:0/3317664972 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62441a6be0 con 0x7f6244102780 2024-09-19T07:32:44.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.240+0000 7f6240c00640 1 -- 172.21.15.159:0/3317664972 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6230005020 con 0x7f6244102780 2024-09-19T07:32:44.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.240+0000 7f6240c00640 1 -- 172.21.15.159:0/3317664972 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62300049e0 con 0x7f6244102780 2024-09-19T07:32:44.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.240+0000 7f624ac00640 1 -- 172.21.15.159:0/3317664972 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62441a7120 con 0x7f6244102780 2024-09-19T07:32:44.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.241+0000 7f624ac00640 1 -- 172.21.15.159:0/3317664972 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62441021b0 con 0x7f6244102780 2024-09-19T07:32:44.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.244+0000 7f6240c00640 1 -- 172.21.15.159:0/3317664972 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f6230005560 con 0x7f6244102780 2024-09-19T07:32:44.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.245+0000 7f6240c00640 1 --2- 172.21.15.159:0/3317664972 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f62280a3350 0x7f62280a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:44.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.245+0000 7f6240c00640 1 -- 172.21.15.159:0/3317664972 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f62300cfff0 con 0x7f6244102780 2024-09-19T07:32:44.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.245+0000 7f6243e00640 1 --2- 172.21.15.159:0/3317664972 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f62280a3350 0x7f62280a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:44.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.246+0000 7f6243e00640 1 --2- 172.21.15.159:0/3317664972 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f62280a3350 0x7f62280a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f62340096c0 tx=0x7f62340046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:44.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.246+0000 7f6240c00640 1 -- 172.21.15.159:0/3317664972 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f62300051d0 con 0x7f6244102780 2024-09-19T07:32:44.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:43 smithi203 ceph-mon[35335]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T07:32:44.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:43 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/2585362973' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T07:32:44.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:43 smithi189 ceph-mon[34851]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T07:32:44.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:43 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/2585362973' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T07:32:44.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.429+0000 7f624ac00640 1 -- 172.21.15.159:0/3317664972 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6244075a20 con 0x7f6244102780 2024-09-19T07:32:44.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.430+0000 7f6240c00640 1 -- 172.21.15.159:0/3317664972 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6244075a20 con 0x7f6244102780 2024-09-19T07:32:44.430 INFO:teuthology.orchestra.run.smithi159.stdout:201863462974 2024-09-19T07:32:44.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.433+0000 7f624ac00640 1 -- 172.21.15.159:0/3317664972 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f62280a3350 msgr2=0x7f62280a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:44.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.433+0000 7f624ac00640 1 --2- 172.21.15.159:0/3317664972 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f62280a3350 0x7f62280a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f62340096c0 tx=0x7f62340046c0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.433+0000 7f624ac00640 1 -- 172.21.15.159:0/3317664972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6244102780 msgr2=0x7f6244074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:44.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.433+0000 7f624ac00640 1 --2- 172.21.15.159:0/3317664972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6244102780 0x7f6244074a30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f623000e9f0 tx=0x7f623000eeb0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.433+0000 7f624ac00640 1 -- 172.21.15.159:0/3317664972 shutdown_connections 2024-09-19T07:32:44.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.433+0000 7f624ac00640 1 --2- 172.21.15.159:0/3317664972 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f62280a3350 0x7f62280a5810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.433+0000 7f624ac00640 1 --2- 172.21.15.159:0/3317664972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6244106f00 0x7f6244074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.433+0000 7f624ac00640 1 --2- 172.21.15.159:0/3317664972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6244102780 0x7f6244074a30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.433+0000 7f624ac00640 1 --2- 172.21.15.159:0/3317664972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6244101dd0 0x7f62440744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:44.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.434+0000 7f624ac00640 1 -- 172.21.15.159:0/3317664972 >> 172.21.15.159:0/3317664972 conn(0x7f62440fd8b0 msgr2=0x7f62440fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:44.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.434+0000 7f624ac00640 1 -- 172.21.15.159:0/3317664972 shutdown_connections 2024-09-19T07:32:44.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.434+0000 7f624ac00640 1 -- 172.21.15.159:0/3317664972 wait complete. 2024-09-19T07:32:45.008 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:44 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/4229433814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T07:32:45.008 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:44 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3317664972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T07:32:45.008 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:44 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2838683172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T07:32:45.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:44 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/4229433814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T07:32:45.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:44 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3317664972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T07:32:45.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:44 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/2838683172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T07:32:45.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:44 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/4229433814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T07:32:45.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:44 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3317664972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T07:32:45.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:44 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/2838683172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T07:32:45.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.294+0000 7f2eb3200640 1 -- 172.21.15.159:0/2516337321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eac102360 msgr2=0x7f2eac10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:45.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.294+0000 7f2eb3200640 1 --2- 172.21.15.159:0/2516337321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eac102360 0x7f2eac10e890 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009cd0 tx=0x7f2ea802a8e0 comp rx=0 tx=0).stop 2024-09-19T07:32:45.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.294+0000 7f2eb3200640 1 -- 172.21.15.159:0/2516337321 shutdown_connections 2024-09-19T07:32:45.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.294+0000 7f2eb3200640 1 --2- 172.21.15.159:0/2516337321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eac102360 0x7f2eac10e890 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:45.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.294+0000 7f2eb3200640 1 --2- 172.21.15.159:0/2516337321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eac1019c0 0x7f2eac101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:45.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.294+0000 7f2eb3200640 1 --2- 172.21.15.159:0/2516337321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eac1079c0 0x7f2eac107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:45.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.294+0000 7f2eb3200640 1 -- 172.21.15.159:0/2516337321 >> 172.21.15.159:0/2516337321 conn(0x7f2eac0fd6e0 msgr2=0x7f2eac0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:45.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.295+0000 7f2eb3200640 1 -- 172.21.15.159:0/2516337321 shutdown_connections 2024-09-19T07:32:45.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.295+0000 7f2eb3200640 1 -- 172.21.15.159:0/2516337321 wait complete. 2024-09-19T07:32:45.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.296+0000 7f2eb3200640 1 Processor -- start 2024-09-19T07:32:45.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.296+0000 7f2eb3200640 1 -- start start 2024-09-19T07:32:45.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.296+0000 7f2eb3200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eac1019c0 0x7f2eac198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:45.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.296+0000 7f2eb3200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eac102360 0x7f2eac1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:45.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.296+0000 7f2eb3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eac1079c0 0x7f2eac19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:45.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.296+0000 7f2eb3200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac111dc0 con 0x7f2eac1079c0 2024-09-19T07:32:45.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.296+0000 7f2eb3200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac111c40 con 0x7f2eac1019c0 2024-09-19T07:32:45.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.296+0000 7f2eb3200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac111f40 con 0x7f2eac102360 2024-09-19T07:32:45.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.297+0000 7f2eb1400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eac102360 0x7f2eac1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:45.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.297+0000 7f2eb2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eac1079c0 0x7f2eac19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:45.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.297+0000 7f2eb1400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eac102360 0x7f2eac1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:56582/0 (socket says 172.21.15.159:56582) 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.297+0000 7f2eb1400640 1 -- 172.21.15.159:0/3499407352 learned_addr learned my addr 172.21.15.159:0/3499407352 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.297+0000 7f2eb1e00640 1 --2- 172.21.15.159:0/3499407352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eac1019c0 0x7f2eac198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.297+0000 7f2eb2800640 1 -- 172.21.15.159:0/3499407352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eac102360 msgr2=0x7f2eac1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.297+0000 7f2eb2800640 1 --2- 172.21.15.159:0/3499407352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eac102360 0x7f2eac1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.297+0000 7f2eb2800640 1 -- 172.21.15.159:0/3499407352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eac1019c0 msgr2=0x7f2eac198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.297+0000 7f2eb2800640 1 --2- 172.21.15.159:0/3499407352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eac1019c0 0x7f2eac198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.297+0000 7f2eb2800640 1 -- 172.21.15.159:0/3499407352 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eac19d210 con 0x7f2eac1079c0 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.298+0000 7f2eb2800640 1 --2- 172.21.15.159:0/3499407352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eac1079c0 0x7f2eac19cb30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2ea802ae00 tx=0x7f2ea8031df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.298+0000 7f2ea6a00640 1 -- 172.21.15.159:0/3499407352 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8003d80 con 0x7f2eac1079c0 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.298+0000 7f2ea6a00640 1 -- 172.21.15.159:0/3499407352 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ea8004520 con 0x7f2eac1079c0 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.298+0000 7f2ea6a00640 1 -- 172.21.15.159:0/3499407352 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8030d00 con 0x7f2eac1079c0 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.298+0000 7f2eb3200640 1 -- 172.21.15.159:0/3499407352 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eac19d4a0 con 0x7f2eac1079c0 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.298+0000 7f2eb3200640 1 -- 172.21.15.159:0/3499407352 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eac1a4de0 con 0x7f2eac1079c0 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.299+0000 7f2eb3200640 1 -- 172.21.15.159:0/3499407352 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eac101e20 con 0x7f2eac1079c0 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.301+0000 7f2ea6a00640 1 -- 172.21.15.159:0/3499407352 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f2ea80046d0 con 0x7f2eac1079c0 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.302+0000 7f2ea6a00640 1 --2- 172.21.15.159:0/3499407352 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2e940a3350 0x7f2e940a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.302+0000 7f2ea6a00640 1 -- 172.21.15.159:0/3499407352 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2ea802e030 con 0x7f2eac1079c0 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.302+0000 7f2eb1e00640 1 --2- 172.21.15.159:0/3499407352 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2e940a3350 0x7f2e940a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.303+0000 7f2eb1e00640 1 --2- 172.21.15.159:0/3499407352 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2e940a3350 0x7f2e940a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009b50 tx=0x7f2ea0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.306+0000 7f2ea6a00640 1 -- 172.21.15.159:0/3499407352 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2ea804b060 con 0x7f2eac1079c0 2024-09-19T07:32:45.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.487+0000 7f2eb3200640 1 -- 172.21.15.159:0/3499407352 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2eac1099c0 con 0x7f2eac1079c0 2024-09-19T07:32:45.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.487+0000 7f2ea6a00640 1 -- 172.21.15.159:0/3499407352 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2ea80bdb30 con 0x7f2eac1079c0 2024-09-19T07:32:45.487 INFO:teuthology.orchestra.run.smithi159.stdout:146028888137 2024-09-19T07:32:45.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.490+0000 7f2eb3200640 1 -- 172.21.15.159:0/3499407352 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2e940a3350 msgr2=0x7f2e940a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:45.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.490+0000 7f2eb3200640 1 --2- 172.21.15.159:0/3499407352 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2e940a3350 0x7f2e940a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009b50 tx=0x7f2ea0005f90 comp rx=0 tx=0).stop 2024-09-19T07:32:45.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.490+0000 7f2eb3200640 1 -- 172.21.15.159:0/3499407352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eac1079c0 msgr2=0x7f2eac19cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:45.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.490+0000 7f2eb3200640 1 --2- 172.21.15.159:0/3499407352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eac1079c0 0x7f2eac19cb30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2ea802ae00 tx=0x7f2ea8031df0 comp rx=0 tx=0).stop 2024-09-19T07:32:45.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.490+0000 7f2eb3200640 1 -- 172.21.15.159:0/3499407352 shutdown_connections 2024-09-19T07:32:45.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.490+0000 7f2eb3200640 1 --2- 172.21.15.159:0/3499407352 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2e940a3350 0x7f2e940a5810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:45.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.490+0000 7f2eb3200640 1 --2- 172.21.15.159:0/3499407352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eac1079c0 0x7f2eac19cb30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:45.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.490+0000 7f2eb3200640 1 --2- 172.21.15.159:0/3499407352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eac102360 0x7f2eac1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:45.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.490+0000 7f2eb3200640 1 --2- 172.21.15.159:0/3499407352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eac1019c0 0x7f2eac198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:45.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.490+0000 7f2eb3200640 1 -- 172.21.15.159:0/3499407352 >> 172.21.15.159:0/3499407352 conn(0x7f2eac0fd6e0 msgr2=0x7f2eac0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:45.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.490+0000 7f2eb3200640 1 -- 172.21.15.159:0/3499407352 shutdown_connections 2024-09-19T07:32:45.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:45.490+0000 7f2eb3200640 1 -- 172.21.15.159:0/3499407352 wait complete. 2024-09-19T07:32:45.907 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:45 smithi159 ceph-mon[30472]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T07:32:45.907 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:45 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3499407352' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T07:32:45.909 INFO:teuthology.orchestra.run.smithi159.stdout:261993005107 2024-09-19T07:32:45.910 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-19T07:32:46.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:45 smithi203 ceph-mon[35335]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T07:32:46.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:45 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3499407352' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T07:32:46.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:45 smithi189 ceph-mon[34851]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T07:32:46.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:45 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3499407352' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T07:32:46.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.491+0000 7fc19e600640 1 -- 172.21.15.159:0/1119669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1981019c0 msgr2=0x7fc198101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:46.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.491+0000 7fc19e600640 1 --2- 172.21.15.159:0/1119669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1981019c0 0x7fc198101e20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc184009930 tx=0x7fc18402a420 comp rx=0 tx=0).stop 2024-09-19T07:32:46.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.491+0000 7fc19e600640 1 -- 172.21.15.159:0/1119669 shutdown_connections 2024-09-19T07:32:46.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.491+0000 7fc19e600640 1 --2- 172.21.15.159:0/1119669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc198102360 0x7fc19810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:46.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.491+0000 7fc19e600640 1 --2- 172.21.15.159:0/1119669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1981019c0 0x7fc198101e20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:46.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.491+0000 7fc19e600640 1 --2- 172.21.15.159:0/1119669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1981079c0 0x7fc198107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:46.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.491+0000 7fc19e600640 1 -- 172.21.15.159:0/1119669 >> 172.21.15.159:0/1119669 conn(0x7fc1980fd740 msgr2=0x7fc1980ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:46.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.491+0000 7fc19e600640 1 -- 172.21.15.159:0/1119669 shutdown_connections 2024-09-19T07:32:46.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.491+0000 7fc19e600640 1 -- 172.21.15.159:0/1119669 wait complete. 2024-09-19T07:32:46.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.492+0000 7fc19e600640 1 Processor -- start 2024-09-19T07:32:46.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.492+0000 7fc19e600640 1 -- start start 2024-09-19T07:32:46.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.493+0000 7fc19e600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1981019c0 0x7fc19819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:46.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.493+0000 7fc19e600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc198102360 0x7fc19819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:46.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.493+0000 7fc19e600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1981079c0 0x7fc19819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:46.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.493+0000 7fc19e600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc198111dc0 con 0x7fc1981079c0 2024-09-19T07:32:46.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.493+0000 7fc19e600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc198111c40 con 0x7fc198102360 2024-09-19T07:32:46.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.493+0000 7fc19e600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc198111f40 con 0x7fc1981019c0 2024-09-19T07:32:46.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.493+0000 7fc19d200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1981019c0 0x7fc19819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:46.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.493+0000 7fc19d200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1981019c0 0x7fc19819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:56568/0 (socket says 172.21.15.159:56568) 2024-09-19T07:32:46.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.493+0000 7fc19d200640 1 -- 172.21.15.159:0/2838683172 learned_addr learned my addr 172.21.15.159:0/2838683172 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:46.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.493+0000 7fc19dc00640 1 --2- 172.21.15.159:0/2838683172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1981079c0 0x7fc19819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:46.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.493+0000 7fc197e00640 1 --2- 172.21.15.159:0/2838683172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc198102360 0x7fc19819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:46.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.493+0000 7fc19d200640 1 -- 172.21.15.159:0/2838683172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc198102360 msgr2=0x7fc19819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:46.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.493+0000 7fc19d200640 1 --2- 172.21.15.159:0/2838683172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc198102360 0x7fc19819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:46.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.493+0000 7fc19d200640 1 -- 172.21.15.159:0/2838683172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1981079c0 msgr2=0x7fc19819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:46.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.493+0000 7fc19d200640 1 --2- 172.21.15.159:0/2838683172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1981079c0 0x7fc19819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:46.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.493+0000 7fc19d200640 1 -- 172.21.15.159:0/2838683172 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19819f490 con 0x7fc1981019c0 2024-09-19T07:32:46.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.493+0000 7fc19dc00640 1 --2- 172.21.15.159:0/2838683172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1981079c0 0x7fc19819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:32:46.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.494+0000 7fc19d200640 1 --2- 172.21.15.159:0/2838683172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1981019c0 0x7fc19819a4e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00e9f0 tx=0x7fc18c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:46.345 INFO:teuthology.orchestra.run.smithi159.stdout:77309411412 2024-09-19T07:32:46.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.494+0000 7fc195600640 1 -- 172.21.15.159:0/2838683172 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00cd80 con 0x7fc1981019c0 2024-09-19T07:32:46.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.494+0000 7fc195600640 1 -- 172.21.15.159:0/2838683172 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc18c010910 con 0x7fc1981019c0 2024-09-19T07:32:46.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.494+0000 7fc19e600640 1 -- 172.21.15.159:0/2838683172 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19819f780 con 0x7fc1981019c0 2024-09-19T07:32:46.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.494+0000 7fc195600640 1 -- 172.21.15.159:0/2838683172 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c018990 con 0x7fc1981019c0 2024-09-19T07:32:46.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.494+0000 7fc19e600640 1 -- 172.21.15.159:0/2838683172 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1981a7060 con 0x7fc1981019c0 2024-09-19T07:32:46.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.495+0000 7fc19e600640 1 -- 172.21.15.159:0/2838683172 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc198101e20 con 0x7fc1981019c0 2024-09-19T07:32:46.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.497+0000 7fc195600640 1 -- 172.21.15.159:0/2838683172 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc18c010ac0 con 0x7fc1981019c0 2024-09-19T07:32:46.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.498+0000 7fc195600640 1 --2- 172.21.15.159:0/2838683172 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc1800a32d0 0x7fc1800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:46.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.499+0000 7fc195600640 1 -- 172.21.15.159:0/2838683172 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc18c0d0540 con 0x7fc1981019c0 2024-09-19T07:32:46.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.499+0000 7fc197e00640 1 --2- 172.21.15.159:0/2838683172 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc1800a32d0 0x7fc1800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:46.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.499+0000 7fc197e00640 1 --2- 172.21.15.159:0/2838683172 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc1800a32d0 0x7fc1800a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc184009900 tx=0x7fc1840058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:46.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.500+0000 7fc195600640 1 -- 172.21.15.159:0/2838683172 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc18c09c6a0 con 0x7fc1981019c0 2024-09-19T07:32:46.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.683+0000 7fc19e600640 1 -- 172.21.15.159:0/2838683172 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc1981a7300 con 0x7fc1981019c0 2024-09-19T07:32:46.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.683+0000 7fc195600640 1 -- 172.21.15.159:0/2838683172 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc18c0a09a0 con 0x7fc1981019c0 2024-09-19T07:32:46.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.686+0000 7fc19e600640 1 -- 172.21.15.159:0/2838683172 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc1800a32d0 msgr2=0x7fc1800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:46.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.686+0000 7fc19e600640 1 --2- 172.21.15.159:0/2838683172 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc1800a32d0 0x7fc1800a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc184009900 tx=0x7fc1840058d0 comp rx=0 tx=0).stop 2024-09-19T07:32:46.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.686+0000 7fc19e600640 1 -- 172.21.15.159:0/2838683172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1981019c0 msgr2=0x7fc19819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:46.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.686+0000 7fc19e600640 1 --2- 172.21.15.159:0/2838683172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1981019c0 0x7fc19819a4e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00e9f0 tx=0x7fc18c00eeb0 comp rx=0 tx=0).stop 2024-09-19T07:32:46.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.686+0000 7fc19e600640 1 -- 172.21.15.159:0/2838683172 shutdown_connections 2024-09-19T07:32:46.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.686+0000 7fc19e600640 1 --2- 172.21.15.159:0/2838683172 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc1800a32d0 0x7fc1800a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:46.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.686+0000 7fc19e600640 1 --2- 172.21.15.159:0/2838683172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1981079c0 0x7fc19819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:46.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.686+0000 7fc19e600640 1 --2- 172.21.15.159:0/2838683172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc198102360 0x7fc19819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:46.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.686+0000 7fc19e600640 1 --2- 172.21.15.159:0/2838683172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1981019c0 0x7fc19819a4e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:46.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.686+0000 7fc19e600640 1 -- 172.21.15.159:0/2838683172 >> 172.21.15.159:0/2838683172 conn(0x7fc1980fd740 msgr2=0x7fc19810cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:46.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.686+0000 7fc19e600640 1 -- 172.21.15.159:0/2838683172 shutdown_connections 2024-09-19T07:32:46.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:44.686+0000 7fc19e600640 1 -- 172.21.15.159:0/2838683172 wait complete. 2024-09-19T07:32:47.253 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738462 got 34359738463 for osd.0 2024-09-19T07:32:47.253 DEBUG:teuthology.parallel:result is None 2024-09-19T07:32:48.045 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:47 smithi159 ceph-mon[30472]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:32:48.057 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:48.095 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411409 got 77309411412 for osd.2 2024-09-19T07:32:48.095 DEBUG:teuthology.parallel:result is None 2024-09-19T07:32:48.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:47 smithi203 ceph-mon[35335]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:32:48.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:47 smithi189 ceph-mon[34851]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:32:48.453 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351007 got 347892351010 for osd.11 2024-09-19T07:32:48.454 DEBUG:teuthology.parallel:result is None 2024-09-19T07:32:49.352 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724609 got 167503724611 for osd.5 2024-09-19T07:32:49.352 DEBUG:teuthology.parallel:result is None 2024-09-19T07:32:49.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.017+0000 7fedace00640 1 -- 172.21.15.159:0/4147629235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feda810e3c0 msgr2=0x7feda8110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:49.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.017+0000 7fedace00640 1 --2- 172.21.15.159:0/4147629235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feda810e3c0 0x7feda8110810 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feda0009ca0 tx=0x7feda002a950 comp rx=0 tx=0).stop 2024-09-19T07:32:49.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.017+0000 7fedace00640 1 -- 172.21.15.159:0/4147629235 shutdown_connections 2024-09-19T07:32:49.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.017+0000 7fedace00640 1 --2- 172.21.15.159:0/4147629235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feda810e3c0 0x7feda8110810 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:49.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.017+0000 7fedace00640 1 --2- 172.21.15.159:0/4147629235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feda8074f40 0x7feda810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:49.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.017+0000 7fedace00640 1 --2- 172.21.15.159:0/4147629235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feda8074620 0x7feda8074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:49.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.017+0000 7fedace00640 1 -- 172.21.15.159:0/4147629235 >> 172.21.15.159:0/4147629235 conn(0x7feda80fd8c0 msgr2=0x7feda80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:49.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.017+0000 7fedace00640 1 -- 172.21.15.159:0/4147629235 shutdown_connections 2024-09-19T07:32:49.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.017+0000 7fedace00640 1 -- 172.21.15.159:0/4147629235 wait complete. 2024-09-19T07:32:49.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.018+0000 7fedace00640 1 Processor -- start 2024-09-19T07:32:49.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.018+0000 7fedace00640 1 -- start start 2024-09-19T07:32:49.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.019+0000 7fedace00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feda8074620 0x7feda819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:49.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.019+0000 7fedace00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feda8074f40 0x7feda819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:49.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.019+0000 7fedace00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feda810e3c0 0x7feda819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:49.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.019+0000 7fedace00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feda8114170 con 0x7feda810e3c0 2024-09-19T07:32:49.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.019+0000 7fedace00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feda8113ff0 con 0x7feda8074620 2024-09-19T07:32:49.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.019+0000 7fedace00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feda81142f0 con 0x7feda8074f40 2024-09-19T07:32:49.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.019+0000 7feda6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feda810e3c0 0x7feda819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:49.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.019+0000 7feda6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feda810e3c0 0x7feda819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39594/0 (socket says 172.21.15.159:39594) 2024-09-19T07:32:49.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.019+0000 7feda6800640 1 -- 172.21.15.159:0/886635038 learned_addr learned my addr 172.21.15.159:0/886635038 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:49.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.019+0000 7fed9fe00640 1 --2- 172.21.15.159:0/886635038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feda8074f40 0x7feda819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:49.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.019+0000 7feda5e00640 1 --2- 172.21.15.159:0/886635038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feda8074620 0x7feda819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:49.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.019+0000 7feda6800640 1 -- 172.21.15.159:0/886635038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feda8074f40 msgr2=0x7feda819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:49.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.019+0000 7feda6800640 1 --2- 172.21.15.159:0/886635038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feda8074f40 0x7feda819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:49.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.019+0000 7feda6800640 1 -- 172.21.15.159:0/886635038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feda8074620 msgr2=0x7feda819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:49.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.019+0000 7feda6800640 1 --2- 172.21.15.159:0/886635038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feda8074620 0x7feda819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:49.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.019+0000 7feda6800640 1 -- 172.21.15.159:0/886635038 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda819f5f0 con 0x7feda810e3c0 2024-09-19T07:32:49.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.019+0000 7fed9fe00640 1 --2- 172.21.15.159:0/886635038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feda8074f40 0x7feda819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:32:49.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.020+0000 7feda6800640 1 --2- 172.21.15.159:0/886635038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feda810e3c0 0x7feda819ef10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feda002ae70 tx=0x7feda002f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:49.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.020+0000 7fed9ea00640 1 -- 172.21.15.159:0/886635038 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0034040 con 0x7feda810e3c0 2024-09-19T07:32:49.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.020+0000 7fed9ea00640 1 -- 172.21.15.159:0/886635038 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feda0030920 con 0x7feda810e3c0 2024-09-19T07:32:49.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.020+0000 7fedace00640 1 -- 172.21.15.159:0/886635038 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda819f880 con 0x7feda810e3c0 2024-09-19T07:32:49.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.020+0000 7fed9ea00640 1 -- 172.21.15.159:0/886635038 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0045480 con 0x7feda810e3c0 2024-09-19T07:32:49.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.020+0000 7fedace00640 1 -- 172.21.15.159:0/886635038 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda81a71c0 con 0x7feda810e3c0 2024-09-19T07:32:49.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.021+0000 7fedace00640 1 -- 172.21.15.159:0/886635038 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda8075cd0 con 0x7feda810e3c0 2024-09-19T07:32:49.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.023+0000 7fed9ea00640 1 -- 172.21.15.159:0/886635038 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7feda0037020 con 0x7feda810e3c0 2024-09-19T07:32:49.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.024+0000 7fed9ea00640 1 --2- 172.21.15.159:0/886635038 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fed8c0a32d0 0x7fed8c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:49.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.025+0000 7fed9ea00640 1 -- 172.21.15.159:0/886635038 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7feda00f0c90 con 0x7feda810e3c0 2024-09-19T07:32:49.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.025+0000 7feda5e00640 1 --2- 172.21.15.159:0/886635038 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fed8c0a32d0 0x7fed8c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:49.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.025+0000 7feda5e00640 1 --2- 172.21.15.159:0/886635038 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fed8c0a32d0 0x7fed8c0a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fed98005d00 tx=0x7fed9800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:49.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.028+0000 7fed9ea00640 1 -- 172.21.15.159:0/886635038 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feda00bd550 con 0x7feda810e3c0 2024-09-19T07:32:49.446 INFO:teuthology.orchestra.run.smithi159.stdout:223338299450 2024-09-19T07:32:49.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.214+0000 7fedace00640 1 -- 172.21.15.159:0/886635038 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7feda8108fb0 con 0x7feda810e3c0 2024-09-19T07:32:49.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.214+0000 7fed9ea00640 1 -- 172.21.15.159:0/886635038 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7feda00bce50 con 0x7feda810e3c0 2024-09-19T07:32:49.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.217+0000 7fedace00640 1 -- 172.21.15.159:0/886635038 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fed8c0a32d0 msgr2=0x7fed8c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:49.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.217+0000 7fedace00640 1 --2- 172.21.15.159:0/886635038 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fed8c0a32d0 0x7fed8c0a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fed98005d00 tx=0x7fed9800b410 comp rx=0 tx=0).stop 2024-09-19T07:32:49.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.217+0000 7fedace00640 1 -- 172.21.15.159:0/886635038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feda810e3c0 msgr2=0x7feda819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:49.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.217+0000 7fedace00640 1 --2- 172.21.15.159:0/886635038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feda810e3c0 0x7feda819ef10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feda002ae70 tx=0x7feda002f5b0 comp rx=0 tx=0).stop 2024-09-19T07:32:49.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.217+0000 7fedace00640 1 -- 172.21.15.159:0/886635038 shutdown_connections 2024-09-19T07:32:49.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.217+0000 7fedace00640 1 --2- 172.21.15.159:0/886635038 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fed8c0a32d0 0x7fed8c0a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:49.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.217+0000 7fedace00640 1 --2- 172.21.15.159:0/886635038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feda810e3c0 0x7feda819ef10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:49.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.217+0000 7fedace00640 1 --2- 172.21.15.159:0/886635038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feda8074f40 0x7feda819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:49.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.217+0000 7fedace00640 1 --2- 172.21.15.159:0/886635038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feda8074620 0x7feda819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:49.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.217+0000 7fedace00640 1 -- 172.21.15.159:0/886635038 >> 172.21.15.159:0/886635038 conn(0x7feda80fd8c0 msgr2=0x7feda80fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:49.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.217+0000 7fedace00640 1 -- 172.21.15.159:0/886635038 shutdown_connections 2024-09-19T07:32:49.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.217+0000 7fedace00640 1 -- 172.21.15.159:0/886635038 wait complete. 2024-09-19T07:32:50.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.891+0000 7fa20a200640 1 -- 172.21.15.159:0/2568525045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa204103e30 msgr2=0x7fa204104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:50.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.891+0000 7fa20a200640 1 --2- 172.21.15.159:0/2568525045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa204103e30 0x7fa204104210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009900 tx=0x7fa1f802a3f0 comp rx=0 tx=0).stop 2024-09-19T07:32:50.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.891+0000 7fa20a200640 1 -- 172.21.15.159:0/2568525045 shutdown_connections 2024-09-19T07:32:50.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.891+0000 7fa20a200640 1 --2- 172.21.15.159:0/2568525045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa204108f60 0x7fa204110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:50.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.891+0000 7fa20a200640 1 --2- 172.21.15.159:0/2568525045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2041047e0 0x7fa204108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:50.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.891+0000 7fa20a200640 1 --2- 172.21.15.159:0/2568525045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa204103e30 0x7fa204104210 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:50.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.891+0000 7fa20a200640 1 -- 172.21.15.159:0/2568525045 >> 172.21.15.159:0/2568525045 conn(0x7fa2040ff8d0 msgr2=0x7fa204101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:50.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.891+0000 7fa20a200640 1 -- 172.21.15.159:0/2568525045 shutdown_connections 2024-09-19T07:32:50.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.891+0000 7fa20a200640 1 -- 172.21.15.159:0/2568525045 wait complete. 2024-09-19T07:32:50.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.892+0000 7fa20a200640 1 Processor -- start 2024-09-19T07:32:50.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.892+0000 7fa20a200640 1 -- start start 2024-09-19T07:32:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.892+0000 7fa20a200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa204103e30 0x7fa20419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.892+0000 7fa20a200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2041047e0 0x7fa20419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.893+0000 7fa20a200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa204108f60 0x7fa20419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.893+0000 7fa20a200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa204114010 con 0x7fa204108f60 2024-09-19T07:32:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.893+0000 7fa20a200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa204113e90 con 0x7fa204103e30 2024-09-19T07:32:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.893+0000 7fa20a200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa204114190 con 0x7fa2041047e0 2024-09-19T07:32:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.893+0000 7fa209800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa204108f60 0x7fa20419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.893+0000 7fa209800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa204108f60 0x7fa20419ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39614/0 (socket says 172.21.15.159:39614) 2024-09-19T07:32:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.893+0000 7fa209800640 1 -- 172.21.15.159:0/3992510086 learned_addr learned my addr 172.21.15.159:0/3992510086 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.893+0000 7fa203e00640 1 --2- 172.21.15.159:0/3992510086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2041047e0 0x7fa20419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.893+0000 7fa209800640 1 -- 172.21.15.159:0/3992510086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2041047e0 msgr2=0x7fa20419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.893+0000 7fa209800640 1 --2- 172.21.15.159:0/3992510086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2041047e0 0x7fa20419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.893+0000 7fa209800640 1 -- 172.21.15.159:0/3992510086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa204103e30 msgr2=0x7fa20419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:50.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.893+0000 7fa209800640 1 --2- 172.21.15.159:0/3992510086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa204103e30 0x7fa20419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:50.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.893+0000 7fa209800640 1 -- 172.21.15.159:0/3992510086 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20419f400 con 0x7fa204108f60 2024-09-19T07:32:50.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.894+0000 7fa209800640 1 --2- 172.21.15.159:0/3992510086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa204108f60 0x7fa20419ed20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00eb10 tx=0x7fa1fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:50.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.894+0000 7fa201600640 1 -- 172.21.15.159:0/3992510086 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc004b20 con 0x7fa204108f60 2024-09-19T07:32:50.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.894+0000 7fa201600640 1 -- 172.21.15.159:0/3992510086 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1fc004cc0 con 0x7fa204108f60 2024-09-19T07:32:50.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.894+0000 7fa20a200640 1 -- 172.21.15.159:0/3992510086 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20419f6f0 con 0x7fa204108f60 2024-09-19T07:32:50.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.894+0000 7fa201600640 1 -- 172.21.15.159:0/3992510086 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc010760 con 0x7fa204108f60 2024-09-19T07:32:50.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.894+0000 7fa20a200640 1 -- 172.21.15.159:0/3992510086 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2041a6fd0 con 0x7fa204108f60 2024-09-19T07:32:50.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.895+0000 7fa20a200640 1 -- 172.21.15.159:0/3992510086 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa204104210 con 0x7fa204108f60 2024-09-19T07:32:50.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.897+0000 7fa201600640 1 -- 172.21.15.159:0/3992510086 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fa1fc005020 con 0x7fa204108f60 2024-09-19T07:32:50.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.898+0000 7fa201600640 1 --2- 172.21.15.159:0/3992510086 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa1ec0a32d0 0x7fa1ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:50.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.899+0000 7fa201600640 1 -- 172.21.15.159:0/3992510086 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa1fc0d05a0 con 0x7fa204108f60 2024-09-19T07:32:50.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.899+0000 7fa208e00640 1 --2- 172.21.15.159:0/3992510086 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa1ec0a32d0 0x7fa1ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:50.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.899+0000 7fa208e00640 1 --2- 172.21.15.159:0/3992510086 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa1ec0a32d0 0x7fa1ec0a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80096c0 tx=0x7fa1f8000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:50.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:49.902+0000 7fa201600640 1 -- 172.21.15.159:0/3992510086 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa1fc09cd50 con 0x7fa204108f60 2024-09-19T07:32:50.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:50.081+0000 7fa20a200640 1 -- 172.21.15.159:0/3992510086 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fa20410bbb0 con 0x7fa204108f60 2024-09-19T07:32:50.082 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:49 smithi159 ceph-mon[30472]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:32:50.082 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:49 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/886635038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T07:32:50.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:50.081+0000 7fa201600640 1 -- 172.21.15.159:0/3992510086 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa1fc09c650 con 0x7fa204108f60 2024-09-19T07:32:50.083 INFO:teuthology.orchestra.run.smithi159.stdout:317827579945 2024-09-19T07:32:50.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:50.084+0000 7fa20a200640 1 -- 172.21.15.159:0/3992510086 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa1ec0a32d0 msgr2=0x7fa1ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:50.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:50.084+0000 7fa20a200640 1 --2- 172.21.15.159:0/3992510086 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa1ec0a32d0 0x7fa1ec0a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80096c0 tx=0x7fa1f8000bc0 comp rx=0 tx=0).stop 2024-09-19T07:32:50.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:50.084+0000 7fa20a200640 1 -- 172.21.15.159:0/3992510086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa204108f60 msgr2=0x7fa20419ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:50.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:50.084+0000 7fa20a200640 1 --2- 172.21.15.159:0/3992510086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa204108f60 0x7fa20419ed20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00eb10 tx=0x7fa1fc00c2d0 comp rx=0 tx=0).stop 2024-09-19T07:32:50.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:50.084+0000 7fa20a200640 1 -- 172.21.15.159:0/3992510086 shutdown_connections 2024-09-19T07:32:50.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:50.084+0000 7fa20a200640 1 --2- 172.21.15.159:0/3992510086 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa1ec0a32d0 0x7fa1ec0a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:50.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:50.084+0000 7fa20a200640 1 --2- 172.21.15.159:0/3992510086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa204108f60 0x7fa20419ed20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:50.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:50.084+0000 7fa20a200640 1 --2- 172.21.15.159:0/3992510086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2041047e0 0x7fa20419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:50.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:50.084+0000 7fa20a200640 1 --2- 172.21.15.159:0/3992510086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa204103e30 0x7fa20419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:50.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:50.084+0000 7fa20a200640 1 -- 172.21.15.159:0/3992510086 >> 172.21.15.159:0/3992510086 conn(0x7fa2040ff8d0 msgr2=0x7fa204100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:50.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:50.084+0000 7fa20a200640 1 -- 172.21.15.159:0/3992510086 shutdown_connections 2024-09-19T07:32:50.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:50.084+0000 7fa20a200640 1 -- 172.21.15.159:0/3992510086 wait complete. 2024-09-19T07:32:50.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:49 smithi203 ceph-mon[35335]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:32:50.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:49 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/886635038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T07:32:50.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:49 smithi189 ceph-mon[34851]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:32:50.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:49 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/886635038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T07:32:50.453 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299445 got 223338299450 for osd.7 2024-09-19T07:32:50.454 DEBUG:teuthology.parallel:result is None 2024-09-19T07:32:51.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:50 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3992510086' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T07:32:51.184 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:51.215 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888134 got 146028888137 for osd.4 2024-09-19T07:32:51.215 DEBUG:teuthology.parallel:result is None 2024-09-19T07:32:51.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:50 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3992510086' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T07:32:51.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:50 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3992510086' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T07:32:51.556 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:52.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.897+0000 7fef03200640 1 -- 172.21.15.159:0/3837751475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef04103e30 msgr2=0x7fef04104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:52.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.897+0000 7fef03200640 1 --2- 172.21.15.159:0/3837751475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef04103e30 0x7fef04104210 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feef4009900 tx=0x7feef402a3f0 comp rx=0 tx=0).stop 2024-09-19T07:32:52.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.897+0000 7fef03200640 1 -- 172.21.15.159:0/3837751475 shutdown_connections 2024-09-19T07:32:52.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.897+0000 7fef03200640 1 --2- 172.21.15.159:0/3837751475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef04108f60 0x7fef04110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:52.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.897+0000 7fef03200640 1 --2- 172.21.15.159:0/3837751475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef041047e0 0x7fef04108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:52.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.897+0000 7fef03200640 1 --2- 172.21.15.159:0/3837751475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef04103e30 0x7fef04104210 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:52.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.897+0000 7fef03200640 1 -- 172.21.15.159:0/3837751475 >> 172.21.15.159:0/3837751475 conn(0x7fef040ff910 msgr2=0x7fef04101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:52.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.897+0000 7fef03200640 1 -- 172.21.15.159:0/3837751475 shutdown_connections 2024-09-19T07:32:52.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.897+0000 7fef03200640 1 -- 172.21.15.159:0/3837751475 wait complete. 2024-09-19T07:32:52.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.898+0000 7fef03200640 1 Processor -- start 2024-09-19T07:32:52.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.899+0000 7fef03200640 1 -- start start 2024-09-19T07:32:52.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.899+0000 7fef03200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef04103e30 0x7fef0419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:52.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.899+0000 7fef03200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef041047e0 0x7fef0419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:52.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.899+0000 7fef03200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef04108f60 0x7fef0419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:52.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.899+0000 7fef03200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef04114010 con 0x7fef041047e0 2024-09-19T07:32:52.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.899+0000 7fef03200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fef04113e90 con 0x7fef04108f60 2024-09-19T07:32:52.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.899+0000 7fef03200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fef04114190 con 0x7fef04103e30 2024-09-19T07:32:52.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.899+0000 7fef01400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef041047e0 0x7fef0419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:52.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.899+0000 7fef01e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef04103e30 0x7fef0419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:52.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.899+0000 7fef01e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef04103e30 0x7fef0419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:56636/0 (socket says 172.21.15.159:56636) 2024-09-19T07:32:52.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.899+0000 7fef01e00640 1 -- 172.21.15.159:0/2562730743 learned_addr learned my addr 172.21.15.159:0/2562730743 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:52.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.899+0000 7fef02800640 1 --2- 172.21.15.159:0/2562730743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef04108f60 0x7fef0419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:52.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.900+0000 7fef01e00640 1 -- 172.21.15.159:0/2562730743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef04108f60 msgr2=0x7fef0419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:52.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.900+0000 7fef01e00640 1 --2- 172.21.15.159:0/2562730743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef04108f60 0x7fef0419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:52.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.900+0000 7fef01e00640 1 -- 172.21.15.159:0/2562730743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef041047e0 msgr2=0x7fef0419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:52.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.900+0000 7fef01e00640 1 --2- 172.21.15.159:0/2562730743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef041047e0 0x7fef0419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:52.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.900+0000 7fef01e00640 1 -- 172.21.15.159:0/2562730743 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0419f3e0 con 0x7fef04103e30 2024-09-19T07:32:52.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.900+0000 7fef01e00640 1 --2- 172.21.15.159:0/2562730743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef04103e30 0x7fef0419a430 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feef4004770 tx=0x7feef4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:52.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.900+0000 7feefaa00640 1 -- 172.21.15.159:0/2562730743 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef403f070 con 0x7fef04103e30 2024-09-19T07:32:52.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.900+0000 7feefaa00640 1 -- 172.21.15.159:0/2562730743 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feef4031ca0 con 0x7fef04103e30 2024-09-19T07:32:52.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.900+0000 7feefaa00640 1 -- 172.21.15.159:0/2562730743 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4035030 con 0x7fef04103e30 2024-09-19T07:32:52.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.900+0000 7fef03200640 1 -- 172.21.15.159:0/2562730743 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0419f670 con 0x7fef04103e30 2024-09-19T07:32:52.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.901+0000 7fef03200640 1 -- 172.21.15.159:0/2562730743 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0419fb00 con 0x7fef04103e30 2024-09-19T07:32:52.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.902+0000 7fef03200640 1 -- 172.21.15.159:0/2562730743 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef04105310 con 0x7fef04103e30 2024-09-19T07:32:52.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.903+0000 7feefaa00640 1 -- 172.21.15.159:0/2562730743 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7feef4042070 con 0x7fef04103e30 2024-09-19T07:32:52.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.904+0000 7feefaa00640 1 --2- 172.21.15.159:0/2562730743 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feee80a3350 0x7feee80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:52.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.905+0000 7feefaa00640 1 -- 172.21.15.159:0/2562730743 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7feef40f03f0 con 0x7fef04103e30 2024-09-19T07:32:52.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.905+0000 7fef01400640 1 --2- 172.21.15.159:0/2562730743 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feee80a3350 0x7feee80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:52.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.905+0000 7fef01400640 1 --2- 172.21.15.159:0/2562730743 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feee80a3350 0x7feee80a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fef040ffd30 tx=0x7feef0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:52.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:51.909+0000 7feefaa00640 1 -- 172.21.15.159:0/2562730743 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feef40bc610 con 0x7fef04103e30 2024-09-19T07:32:52.051 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:51 smithi159 ceph-mon[30472]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:32:52.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:52.087+0000 7fef03200640 1 -- 172.21.15.159:0/2562730743 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fef04104210 con 0x7fef04103e30 2024-09-19T07:32:52.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:52.088+0000 7feefaa00640 1 -- 172.21.15.159:0/2562730743 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feef40c0910 con 0x7fef04103e30 2024-09-19T07:32:52.089 INFO:teuthology.orchestra.run.smithi159.stdout:55834574938 2024-09-19T07:32:52.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:52.090+0000 7fef03200640 1 -- 172.21.15.159:0/2562730743 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feee80a3350 msgr2=0x7feee80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:52.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:52.090+0000 7fef03200640 1 --2- 172.21.15.159:0/2562730743 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feee80a3350 0x7feee80a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fef040ffd30 tx=0x7feef0005f90 comp rx=0 tx=0).stop 2024-09-19T07:32:52.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:52.090+0000 7fef03200640 1 -- 172.21.15.159:0/2562730743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef04103e30 msgr2=0x7fef0419a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:52.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:52.090+0000 7fef03200640 1 --2- 172.21.15.159:0/2562730743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef04103e30 0x7fef0419a430 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feef4004770 tx=0x7feef4003c10 comp rx=0 tx=0).stop 2024-09-19T07:32:52.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:52.090+0000 7fef03200640 1 -- 172.21.15.159:0/2562730743 shutdown_connections 2024-09-19T07:32:52.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:52.090+0000 7fef03200640 1 --2- 172.21.15.159:0/2562730743 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feee80a3350 0x7feee80a5810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:52.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:52.090+0000 7fef03200640 1 --2- 172.21.15.159:0/2562730743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef04108f60 0x7fef0419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:52.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:52.090+0000 7fef03200640 1 --2- 172.21.15.159:0/2562730743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef041047e0 0x7fef0419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:52.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:52.090+0000 7fef03200640 1 --2- 172.21.15.159:0/2562730743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef04103e30 0x7fef0419a430 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:52.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:52.090+0000 7fef03200640 1 -- 172.21.15.159:0/2562730743 >> 172.21.15.159:0/2562730743 conn(0x7fef040ff910 msgr2=0x7fef04100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:52.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:52.090+0000 7fef03200640 1 -- 172.21.15.159:0/2562730743 shutdown_connections 2024-09-19T07:32:52.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:52.090+0000 7fef03200640 1 -- 172.21.15.159:0/2562730743 wait complete. 2024-09-19T07:32:52.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:51 smithi203 ceph-mon[35335]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:32:52.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:51 smithi189 ceph-mon[34851]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:32:52.479 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462971 got 201863462974 for osd.6 2024-09-19T07:32:52.479 DEBUG:teuthology.parallel:result is None 2024-09-19T07:32:52.840 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:53.069 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:52 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2562730743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T07:32:53.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.234+0000 7f8fd6200640 1 -- 172.21.15.159:0/210934453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fd0103ff0 msgr2=0x7f8fd01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:53.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.234+0000 7f8fd6200640 1 --2- 172.21.15.159:0/210934453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fd0103ff0 0x7f8fd01043d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009900 tx=0x7f8fc002a410 comp rx=0 tx=0).stop 2024-09-19T07:32:53.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.235+0000 7f8fd6200640 1 -- 172.21.15.159:0/210934453 shutdown_connections 2024-09-19T07:32:53.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.235+0000 7f8fd6200640 1 --2- 172.21.15.159:0/210934453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fd0109120 0x7f8fd0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:53.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.235+0000 7f8fd6200640 1 --2- 172.21.15.159:0/210934453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd01049a0 0x7f8fd01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:53.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.235+0000 7f8fd6200640 1 --2- 172.21.15.159:0/210934453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fd0103ff0 0x7f8fd01043d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:53.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.235+0000 7f8fd6200640 1 -- 172.21.15.159:0/210934453 >> 172.21.15.159:0/210934453 conn(0x7f8fd00ffa90 msgr2=0x7f8fd0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:53.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.235+0000 7f8fd6200640 1 -- 172.21.15.159:0/210934453 shutdown_connections 2024-09-19T07:32:53.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.235+0000 7f8fd6200640 1 -- 172.21.15.159:0/210934453 wait complete. 2024-09-19T07:32:53.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.236+0000 7f8fd6200640 1 Processor -- start 2024-09-19T07:32:53.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.236+0000 7f8fd6200640 1 -- start start 2024-09-19T07:32:53.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.236+0000 7f8fd6200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fd0103ff0 0x7f8fd019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:53.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.236+0000 7f8fd6200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fd01049a0 0x7f8fd019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:53.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.236+0000 7f8fd6200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd0109120 0x7f8fd019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:53.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.236+0000 7f8fd6200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0114150 con 0x7f8fd01049a0 2024-09-19T07:32:53.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.236+0000 7f8fd6200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0113fd0 con 0x7f8fd0109120 2024-09-19T07:32:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.236+0000 7f8fd6200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01142d0 con 0x7f8fd0103ff0 2024-09-19T07:32:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.237+0000 7f8fcea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fd01049a0 0x7f8fd019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.237+0000 7f8fcf400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fd0103ff0 0x7f8fd019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.237+0000 7f8fcea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fd01049a0 0x7f8fd019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44618/0 (socket says 172.21.15.159:44618) 2024-09-19T07:32:53.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.237+0000 7f8fcea00640 1 -- 172.21.15.159:0/1166237141 learned_addr learned my addr 172.21.15.159:0/1166237141 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:53.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.237+0000 7f8fcfe00640 1 --2- 172.21.15.159:0/1166237141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd0109120 0x7f8fd019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:53.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.237+0000 7f8fcea00640 1 -- 172.21.15.159:0/1166237141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fd0103ff0 msgr2=0x7f8fd019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:53.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.237+0000 7f8fcea00640 1 --2- 172.21.15.159:0/1166237141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fd0103ff0 0x7f8fd019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:53.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.237+0000 7f8fcea00640 1 -- 172.21.15.159:0/1166237141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd0109120 msgr2=0x7f8fd019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:53.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.237+0000 7f8fcea00640 1 --2- 172.21.15.159:0/1166237141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd0109120 0x7f8fd019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:53.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.237+0000 7f8fcea00640 1 -- 172.21.15.159:0/1166237141 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd019f5e0 con 0x7f8fd01049a0 2024-09-19T07:32:53.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.237+0000 7f8fcf400640 1 --2- 172.21.15.159:0/1166237141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fd0103ff0 0x7f8fd019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:32:53.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.237+0000 7f8fcfe00640 1 --2- 172.21.15.159:0/1166237141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd0109120 0x7f8fd019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:32:53.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.237+0000 7f8fcea00640 1 --2- 172.21.15.159:0/1166237141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fd01049a0 0x7f8fd019ab70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00e9f0 tx=0x7f8fbc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:53.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.238+0000 7f8fc7e00640 1 -- 172.21.15.159:0/1166237141 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc00cd80 con 0x7f8fd01049a0 2024-09-19T07:32:53.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.238+0000 7f8fc7e00640 1 -- 172.21.15.159:0/1166237141 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fbc010910 con 0x7f8fd01049a0 2024-09-19T07:32:53.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.238+0000 7f8fc7e00640 1 -- 172.21.15.159:0/1166237141 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc018990 con 0x7f8fd01049a0 2024-09-19T07:32:53.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.238+0000 7f8fd6200640 1 -- 172.21.15.159:0/1166237141 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd019f8d0 con 0x7f8fd01049a0 2024-09-19T07:32:53.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.238+0000 7f8fd6200640 1 -- 172.21.15.159:0/1166237141 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd01a71b0 con 0x7f8fd01049a0 2024-09-19T07:32:53.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.239+0000 7f8fd6200640 1 -- 172.21.15.159:0/1166237141 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd01043d0 con 0x7f8fd01049a0 2024-09-19T07:32:53.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.242+0000 7f8fc7e00640 1 -- 172.21.15.159:0/1166237141 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8fbc010ac0 con 0x7f8fd01049a0 2024-09-19T07:32:53.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.243+0000 7f8fc7e00640 1 --2- 172.21.15.159:0/1166237141 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8fb40a3350 0x7f8fb40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:53.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.243+0000 7f8fc7e00640 1 -- 172.21.15.159:0/1166237141 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f8fbc0d0560 con 0x7f8fd01049a0 2024-09-19T07:32:53.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.243+0000 7f8fcf400640 1 --2- 172.21.15.159:0/1166237141 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8fb40a3350 0x7f8fb40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:53.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.244+0000 7f8fcf400640 1 --2- 172.21.15.159:0/1166237141 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8fb40a3350 0x7f8fb40a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009230 tx=0x7f8fc00058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:53.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.249+0000 7f8fc7e00640 1 -- 172.21.15.159:0/1166237141 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8fbc09c6c0 con 0x7f8fd01049a0 2024-09-19T07:32:53.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:52 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/2562730743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T07:32:53.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:52 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/2562730743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T07:32:53.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.438+0000 7f8fd6200640 1 -- 172.21.15.159:0/1166237141 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f8fd002d7c0 con 0x7f8fd01049a0 2024-09-19T07:32:53.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.438+0000 7f8fc7e00640 1 -- 172.21.15.159:0/1166237141 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8fbc0a09c0 con 0x7f8fd01049a0 2024-09-19T07:32:53.439 INFO:teuthology.orchestra.run.smithi159.stdout:296352743472 2024-09-19T07:32:53.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.441+0000 7f8fd6200640 1 -- 172.21.15.159:0/1166237141 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8fb40a3350 msgr2=0x7f8fb40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:53.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.441+0000 7f8fd6200640 1 --2- 172.21.15.159:0/1166237141 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8fb40a3350 0x7f8fb40a5810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009230 tx=0x7f8fc00058d0 comp rx=0 tx=0).stop 2024-09-19T07:32:53.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.441+0000 7f8fd6200640 1 -- 172.21.15.159:0/1166237141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fd01049a0 msgr2=0x7f8fd019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:53.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.441+0000 7f8fd6200640 1 --2- 172.21.15.159:0/1166237141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fd01049a0 0x7f8fd019ab70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00e9f0 tx=0x7f8fbc00eeb0 comp rx=0 tx=0).stop 2024-09-19T07:32:53.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.441+0000 7f8fd6200640 1 -- 172.21.15.159:0/1166237141 shutdown_connections 2024-09-19T07:32:53.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.441+0000 7f8fd6200640 1 --2- 172.21.15.159:0/1166237141 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8fb40a3350 0x7f8fb40a5810 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:53.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.441+0000 7f8fd6200640 1 --2- 172.21.15.159:0/1166237141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd0109120 0x7f8fd019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:53.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.441+0000 7f8fd6200640 1 --2- 172.21.15.159:0/1166237141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fd01049a0 0x7f8fd019ab70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:53.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.441+0000 7f8fd6200640 1 --2- 172.21.15.159:0/1166237141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fd0103ff0 0x7f8fd019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:53.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.441+0000 7f8fd6200640 1 -- 172.21.15.159:0/1166237141 >> 172.21.15.159:0/1166237141 conn(0x7f8fd00ffa90 msgr2=0x7f8fd0100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:53.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.441+0000 7f8fd6200640 1 -- 172.21.15.159:0/1166237141 shutdown_connections 2024-09-19T07:32:53.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.441+0000 7f8fd6200640 1 -- 172.21.15.159:0/1166237141 wait complete. 2024-09-19T07:32:53.933 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:53 smithi159 ceph-mon[30472]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:32:53.933 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:53 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1166237141' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T07:32:53.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.944+0000 7ff02c000640 1 -- 172.21.15.159:0/1852345267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024074f40 msgr2=0x7ff02410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:53.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.944+0000 7ff02c000640 1 --2- 172.21.15.159:0/1852345267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024074f40 0x7ff02410de80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff010009900 tx=0x7ff01002a480 comp rx=0 tx=0).stop 2024-09-19T07:32:53.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.944+0000 7ff02c000640 1 -- 172.21.15.159:0/1852345267 shutdown_connections 2024-09-19T07:32:53.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.944+0000 7ff02c000640 1 --2- 172.21.15.159:0/1852345267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff02410e3c0 0x7ff024110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:53.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.944+0000 7ff02c000640 1 --2- 172.21.15.159:0/1852345267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024074f40 0x7ff02410de80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:53.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.944+0000 7ff02c000640 1 --2- 172.21.15.159:0/1852345267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff024074620 0x7ff024074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:53.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.944+0000 7ff02c000640 1 -- 172.21.15.159:0/1852345267 >> 172.21.15.159:0/1852345267 conn(0x7ff0240fd8c0 msgr2=0x7ff0240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:53.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.944+0000 7ff02c000640 1 -- 172.21.15.159:0/1852345267 shutdown_connections 2024-09-19T07:32:53.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.945+0000 7ff02c000640 1 -- 172.21.15.159:0/1852345267 wait complete. 2024-09-19T07:32:53.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.945+0000 7ff02c000640 1 Processor -- start 2024-09-19T07:32:53.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.946+0000 7ff02c000640 1 -- start start 2024-09-19T07:32:53.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.946+0000 7ff02c000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff024074620 0x7ff02419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:53.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.946+0000 7ff02c000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024074f40 0x7ff02419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:53.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.946+0000 7ff02c000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff02410e3c0 0x7ff02419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:53.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.946+0000 7ff02c000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff024114140 con 0x7ff02410e3c0 2024-09-19T07:32:53.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.946+0000 7ff02c000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff024113fc0 con 0x7ff024074f40 2024-09-19T07:32:53.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.946+0000 7ff02c000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff0241142c0 con 0x7ff024074620 2024-09-19T07:32:53.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.946+0000 7ff029600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff024074620 0x7ff02419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:53.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.946+0000 7ff02a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff02410e3c0 0x7ff02419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:53.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.946+0000 7ff029600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff024074620 0x7ff02419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:51690/0 (socket says 172.21.15.159:51690) 2024-09-19T07:32:53.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.946+0000 7ff029600640 1 -- 172.21.15.159:0/1523651114 learned_addr learned my addr 172.21.15.159:0/1523651114 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:53.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.947+0000 7ff028c00640 1 --2- 172.21.15.159:0/1523651114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024074f40 0x7ff02419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:53.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.947+0000 7ff029600640 1 -- 172.21.15.159:0/1523651114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024074f40 msgr2=0x7ff02419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:53.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.947+0000 7ff029600640 1 --2- 172.21.15.159:0/1523651114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024074f40 0x7ff02419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:53.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.947+0000 7ff029600640 1 -- 172.21.15.159:0/1523651114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff02410e3c0 msgr2=0x7ff02419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:53.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.947+0000 7ff029600640 1 --2- 172.21.15.159:0/1523651114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff02410e3c0 0x7ff02419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:53.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.947+0000 7ff029600640 1 -- 172.21.15.159:0/1523651114 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02419f610 con 0x7ff024074620 2024-09-19T07:32:53.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.947+0000 7ff02a000640 1 --2- 172.21.15.159:0/1523651114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff02410e3c0 0x7ff02419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:32:53.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.947+0000 7ff028c00640 1 --2- 172.21.15.159:0/1523651114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024074f40 0x7ff02419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:32:53.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.947+0000 7ff029600640 1 --2- 172.21.15.159:0/1523651114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff024074620 0x7ff02419a660 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff01800e9f0 tx=0x7ff01800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:53.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.947+0000 7ff022000640 1 -- 172.21.15.159:0/1523651114 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01800ccc0 con 0x7ff024074620 2024-09-19T07:32:53.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.947+0000 7ff022000640 1 -- 172.21.15.159:0/1523651114 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff018010910 con 0x7ff024074620 2024-09-19T07:32:53.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.948+0000 7ff02c000640 1 -- 172.21.15.159:0/1523651114 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02419f900 con 0x7ff024074620 2024-09-19T07:32:53.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.948+0000 7ff022000640 1 -- 172.21.15.159:0/1523651114 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0180188a0 con 0x7ff024074620 2024-09-19T07:32:53.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.948+0000 7ff02c000640 1 -- 172.21.15.159:0/1523651114 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0241a71e0 con 0x7ff024074620 2024-09-19T07:32:53.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.949+0000 7ff02c000640 1 -- 172.21.15.159:0/1523651114 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff024074a00 con 0x7ff024074620 2024-09-19T07:32:53.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.951+0000 7ff022000640 1 -- 172.21.15.159:0/1523651114 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff018010ac0 con 0x7ff024074620 2024-09-19T07:32:53.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.952+0000 7ff022000640 1 --2- 172.21.15.159:0/1523651114 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff00c0a3350 0x7ff00c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:53.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.952+0000 7ff022000640 1 -- 172.21.15.159:0/1523651114 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ff0180d0440 con 0x7ff024074620 2024-09-19T07:32:53.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.952+0000 7ff028c00640 1 --2- 172.21.15.159:0/1523651114 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff00c0a3350 0x7ff00c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:53.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.953+0000 7ff028c00640 1 --2- 172.21.15.159:0/1523651114 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff00c0a3350 0x7ff00c0a5810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff0240ffec0 tx=0x7ff010005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:53.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:53.955+0000 7ff022000640 1 -- 172.21.15.159:0/1523651114 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff01809cbf0 con 0x7ff024074620 2024-09-19T07:32:54.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:54.136+0000 7ff02c000640 1 -- 172.21.15.159:0/1523651114 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7ff02419b9c0 con 0x7ff024074620 2024-09-19T07:32:54.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:54.137+0000 7ff022000640 1 -- 172.21.15.159:0/1523651114 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff02419b9c0 con 0x7ff024074620 2024-09-19T07:32:54.138 INFO:teuthology.orchestra.run.smithi159.stdout:261993005109 2024-09-19T07:32:54.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:54.139+0000 7ff02c000640 1 -- 172.21.15.159:0/1523651114 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff00c0a3350 msgr2=0x7ff00c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:54.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:54.139+0000 7ff02c000640 1 --2- 172.21.15.159:0/1523651114 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff00c0a3350 0x7ff00c0a5810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff0240ffec0 tx=0x7ff010005a20 comp rx=0 tx=0).stop 2024-09-19T07:32:54.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:54.139+0000 7ff02c000640 1 -- 172.21.15.159:0/1523651114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff024074620 msgr2=0x7ff02419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:54.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:54.139+0000 7ff02c000640 1 --2- 172.21.15.159:0/1523651114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff024074620 0x7ff02419a660 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff01800e9f0 tx=0x7ff01800edf0 comp rx=0 tx=0).stop 2024-09-19T07:32:54.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:54.139+0000 7ff02c000640 1 -- 172.21.15.159:0/1523651114 shutdown_connections 2024-09-19T07:32:54.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:54.139+0000 7ff02c000640 1 --2- 172.21.15.159:0/1523651114 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ff00c0a3350 0x7ff00c0a5810 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:54.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:54.139+0000 7ff02c000640 1 --2- 172.21.15.159:0/1523651114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff02410e3c0 0x7ff02419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:54.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:54.139+0000 7ff02c000640 1 --2- 172.21.15.159:0/1523651114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024074f40 0x7ff02419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:54.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:54.139+0000 7ff02c000640 1 --2- 172.21.15.159:0/1523651114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff024074620 0x7ff02419a660 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:54.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:54.139+0000 7ff02c000640 1 -- 172.21.15.159:0/1523651114 >> 172.21.15.159:0/1523651114 conn(0x7ff0240fd8c0 msgr2=0x7ff0240ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:54.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:54.139+0000 7ff02c000640 1 -- 172.21.15.159:0/1523651114 shutdown_connections 2024-09-19T07:32:54.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:54.139+0000 7ff02c000640 1 -- 172.21.15.159:0/1523651114 wait complete. 2024-09-19T07:32:54.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:53 smithi203 ceph-mon[35335]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:32:54.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:53 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1166237141' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T07:32:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:53 smithi189 ceph-mon[34851]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:32:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:53 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1166237141' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T07:32:54.525 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579941 got 317827579945 for osd.10 2024-09-19T07:32:54.526 DEBUG:teuthology.parallel:result is None 2024-09-19T07:32:55.040 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:54 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1523651114' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T07:32:55.055 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:55.081 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574933 got 55834574938 for osd.1 2024-09-19T07:32:55.082 DEBUG:teuthology.parallel:result is None 2024-09-19T07:32:55.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:54 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1523651114' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T07:32:55.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:54 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1523651114' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T07:32:56.026 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743468 got 296352743472 for osd.9 2024-09-19T07:32:56.026 DEBUG:teuthology.parallel:result is None 2024-09-19T07:32:56.207 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:55 smithi189 ceph-mon[34851]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:32:56.273 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:55 smithi159 ceph-mon[30472]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:32:56.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:55 smithi203 ceph-mon[35335]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:32:56.613 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005107 got 261993005109 for osd.8 2024-09-19T07:32:56.613 DEBUG:teuthology.parallel:result is None 2024-09-19T07:32:56.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.837+0000 7f4576e00640 1 -- 172.21.15.159:0/4069919802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45701049a0 msgr2=0x7f45701089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:56.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.837+0000 7f4576e00640 1 --2- 172.21.15.159:0/4069919802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45701049a0 0x7f45701089f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4558009900 tx=0x7f455802a3f0 comp rx=0 tx=0).stop 2024-09-19T07:32:56.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.837+0000 7f4576e00640 1 -- 172.21.15.159:0/4069919802 shutdown_connections 2024-09-19T07:32:56.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.837+0000 7f4576e00640 1 --2- 172.21.15.159:0/4069919802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4570109120 0x7f4570110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:56.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.837+0000 7f4576e00640 1 --2- 172.21.15.159:0/4069919802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45701049a0 0x7f45701089f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:56.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.837+0000 7f4576e00640 1 --2- 172.21.15.159:0/4069919802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4570103ff0 0x7f45701043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:56.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.837+0000 7f4576e00640 1 -- 172.21.15.159:0/4069919802 >> 172.21.15.159:0/4069919802 conn(0x7f45700ffa90 msgr2=0x7f4570101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:56.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.837+0000 7f4576e00640 1 -- 172.21.15.159:0/4069919802 shutdown_connections 2024-09-19T07:32:56.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.837+0000 7f4576e00640 1 -- 172.21.15.159:0/4069919802 wait complete. 2024-09-19T07:32:56.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.838+0000 7f4576e00640 1 Processor -- start 2024-09-19T07:32:56.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.838+0000 7f4576e00640 1 -- start start 2024-09-19T07:32:56.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.839+0000 7f4576e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4570103ff0 0x7f457019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:56.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.839+0000 7f4576e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45701049a0 0x7f457019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:56.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.839+0000 7f4576e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4570109120 0x7f457019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:56.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.839+0000 7f4576e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4570114170 con 0x7f45701049a0 2024-09-19T07:32:56.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.839+0000 7f4576e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4570113ff0 con 0x7f4570109120 2024-09-19T07:32:56.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.839+0000 7f4576e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f45701142f0 con 0x7f4570103ff0 2024-09-19T07:32:56.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.839+0000 7f4567400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45701049a0 0x7f457019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:56.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.839+0000 7f4567400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45701049a0 0x7f457019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44650/0 (socket says 172.21.15.159:44650) 2024-09-19T07:32:56.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.839+0000 7f4567400640 1 -- 172.21.15.159:0/67301874 learned_addr learned my addr 172.21.15.159:0/67301874 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:56.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.839+0000 7f456fe00640 1 --2- 172.21.15.159:0/67301874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4570103ff0 0x7f457019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:56.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.840+0000 7f4567400640 1 -- 172.21.15.159:0/67301874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4570103ff0 msgr2=0x7f457019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:56.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.840+0000 7f4567400640 1 --2- 172.21.15.159:0/67301874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4570103ff0 0x7f457019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:56.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.840+0000 7f4567400640 1 -- 172.21.15.159:0/67301874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4570109120 msgr2=0x7f457019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:56.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.840+0000 7f4567400640 1 --2- 172.21.15.159:0/67301874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4570109120 0x7f457019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:56.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.840+0000 7f4567400640 1 -- 172.21.15.159:0/67301874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457019f5e0 con 0x7f45701049a0 2024-09-19T07:32:56.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.840+0000 7f4567400640 1 --2- 172.21.15.159:0/67301874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45701049a0 0x7f457019ab70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4558004770 tx=0x7f4558003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:56.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.840+0000 7f456d600640 1 -- 172.21.15.159:0/67301874 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455803f070 con 0x7f45701049a0 2024-09-19T07:32:56.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.840+0000 7f456d600640 1 -- 172.21.15.159:0/67301874 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4558031ca0 con 0x7f45701049a0 2024-09-19T07:32:56.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.840+0000 7f4576e00640 1 -- 172.21.15.159:0/67301874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457019f870 con 0x7f45701049a0 2024-09-19T07:32:56.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.840+0000 7f456d600640 1 -- 172.21.15.159:0/67301874 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558035030 con 0x7f45701049a0 2024-09-19T07:32:56.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.840+0000 7f4576e00640 1 -- 172.21.15.159:0/67301874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45701a71b0 con 0x7f45701049a0 2024-09-19T07:32:56.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.842+0000 7f4576e00640 1 -- 172.21.15.159:0/67301874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45701043d0 con 0x7f45701049a0 2024-09-19T07:32:56.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.843+0000 7f456d600640 1 -- 172.21.15.159:0/67301874 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4558042070 con 0x7f45701049a0 2024-09-19T07:32:56.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.845+0000 7f456d600640 1 --2- 172.21.15.159:0/67301874 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f45540a3350 0x7f45540a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:56.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.846+0000 7f456d600640 1 -- 172.21.15.159:0/67301874 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f45580f0530 con 0x7f45701049a0 2024-09-19T07:32:56.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.846+0000 7f456fe00640 1 --2- 172.21.15.159:0/67301874 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f45540a3350 0x7f45540a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:56.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.846+0000 7f456fe00640 1 --2- 172.21.15.159:0/67301874 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f45540a3350 0x7f45540a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f456000bbe0 tx=0x7f456000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:56.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:56.850+0000 7f456d600640 1 -- 172.21.15.159:0/67301874 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f45580bcce0 con 0x7f45701049a0 2024-09-19T07:32:57.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:57.027+0000 7f4576e00640 1 -- 172.21.15.159:0/67301874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f457002d7c0 con 0x7f45701049a0 2024-09-19T07:32:57.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:57.027+0000 7f456d600640 1 -- 172.21.15.159:0/67301874 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f45580bc5e0 con 0x7f45701049a0 2024-09-19T07:32:57.028 INFO:teuthology.orchestra.run.smithi159.stdout:111669149776 2024-09-19T07:32:57.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:57.029+0000 7f4576e00640 1 -- 172.21.15.159:0/67301874 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f45540a3350 msgr2=0x7f45540a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:57.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:57.029+0000 7f4576e00640 1 --2- 172.21.15.159:0/67301874 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f45540a3350 0x7f45540a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f456000bbe0 tx=0x7f456000b390 comp rx=0 tx=0).stop 2024-09-19T07:32:57.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:57.029+0000 7f4576e00640 1 -- 172.21.15.159:0/67301874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45701049a0 msgr2=0x7f457019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:57.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:57.029+0000 7f4576e00640 1 --2- 172.21.15.159:0/67301874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45701049a0 0x7f457019ab70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4558004770 tx=0x7f4558003c10 comp rx=0 tx=0).stop 2024-09-19T07:32:57.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:57.029+0000 7f4576e00640 1 -- 172.21.15.159:0/67301874 shutdown_connections 2024-09-19T07:32:57.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:57.029+0000 7f4576e00640 1 --2- 172.21.15.159:0/67301874 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f45540a3350 0x7f45540a5810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:57.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:57.029+0000 7f4576e00640 1 --2- 172.21.15.159:0/67301874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4570109120 0x7f457019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:57.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:57.029+0000 7f4576e00640 1 --2- 172.21.15.159:0/67301874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45701049a0 0x7f457019ab70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:57.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:57.029+0000 7f4576e00640 1 --2- 172.21.15.159:0/67301874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4570103ff0 0x7f457019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:57.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:57.029+0000 7f4576e00640 1 -- 172.21.15.159:0/67301874 >> 172.21.15.159:0/67301874 conn(0x7f45700ffa90 msgr2=0x7f4570100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:57.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:57.030+0000 7f4576e00640 1 -- 172.21.15.159:0/67301874 shutdown_connections 2024-09-19T07:32:57.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:57.030+0000 7f4576e00640 1 -- 172.21.15.159:0/67301874 wait complete. 2024-09-19T07:32:57.853 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149773 got 111669149776 for osd.3 2024-09-19T07:32:57.853 DEBUG:teuthology.parallel:result is None 2024-09-19T07:32:57.853 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T07:32:57.853 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T07:32:58.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:57 smithi159 ceph-mon[30472]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:32:58.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:57 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/67301874' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T07:32:58.227 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:57 smithi203 ceph-mon[35335]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:32:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:57 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/67301874' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T07:32:58.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:57 smithi189 ceph-mon[34851]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:32:58.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:57 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/67301874' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T07:32:58.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.881+0000 7ffbcf000640 1 -- 172.21.15.159:0/4145351471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbc8101de0 msgr2=0x7ffbc81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:58.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.881+0000 7ffbcf000640 1 --2- 172.21.15.159:0/4145351471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbc8101de0 0x7ffbc81021c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009900 tx=0x7ffbb802a3f0 comp rx=0 tx=0).stop 2024-09-19T07:32:58.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.882+0000 7ffbcf000640 1 -- 172.21.15.159:0/4145351471 shutdown_connections 2024-09-19T07:32:58.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.882+0000 7ffbcf000640 1 --2- 172.21.15.159:0/4145351471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbc8106f10 0x7ffbc810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:58.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.882+0000 7ffbcf000640 1 --2- 172.21.15.159:0/4145351471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbc8102790 0x7ffbc81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:58.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.882+0000 7ffbcf000640 1 --2- 172.21.15.159:0/4145351471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbc8101de0 0x7ffbc81021c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:58.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.882+0000 7ffbcf000640 1 -- 172.21.15.159:0/4145351471 >> 172.21.15.159:0/4145351471 conn(0x7ffbc80fd8c0 msgr2=0x7ffbc80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:58.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.882+0000 7ffbcf000640 1 -- 172.21.15.159:0/4145351471 shutdown_connections 2024-09-19T07:32:58.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.882+0000 7ffbcf000640 1 -- 172.21.15.159:0/4145351471 wait complete. 2024-09-19T07:32:58.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.883+0000 7ffbcf000640 1 Processor -- start 2024-09-19T07:32:58.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.883+0000 7ffbcf000640 1 -- start start 2024-09-19T07:32:58.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.883+0000 7ffbcf000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbc8101de0 0x7ffbc819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:58.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.883+0000 7ffbcf000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbc8102790 0x7ffbc819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:58.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.883+0000 7ffbcf000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbc8106f10 0x7ffbc819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:58.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.883+0000 7ffbcf000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc8111f70 con 0x7ffbc8101de0 2024-09-19T07:32:58.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.883+0000 7ffbcf000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc8111df0 con 0x7ffbc8102790 2024-09-19T07:32:58.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.883+0000 7ffbcf000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc81120f0 con 0x7ffbc8106f10 2024-09-19T07:32:58.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.883+0000 7ffbc7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbc8101de0 0x7ffbc819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:58.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.884+0000 7ffbc7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbc8101de0 0x7ffbc819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44670/0 (socket says 172.21.15.159:44670) 2024-09-19T07:32:58.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.884+0000 7ffbc7e00640 1 -- 172.21.15.159:0/2825301108 learned_addr learned my addr 172.21.15.159:0/2825301108 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:32:58.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.884+0000 7ffbbf400640 1 --2- 172.21.15.159:0/2825301108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbc8102790 0x7ffbc819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:58.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.884+0000 7ffbcd000640 1 --2- 172.21.15.159:0/2825301108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbc8106f10 0x7ffbc819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:58.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.884+0000 7ffbc7e00640 1 -- 172.21.15.159:0/2825301108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbc8106f10 msgr2=0x7ffbc819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:58.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.884+0000 7ffbc7e00640 1 --2- 172.21.15.159:0/2825301108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbc8106f10 0x7ffbc819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:58.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.884+0000 7ffbc7e00640 1 -- 172.21.15.159:0/2825301108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbc8102790 msgr2=0x7ffbc819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:58.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.884+0000 7ffbc7e00640 1 --2- 172.21.15.159:0/2825301108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbc8102790 0x7ffbc819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:58.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.884+0000 7ffbc7e00640 1 -- 172.21.15.159:0/2825301108 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc819f5b0 con 0x7ffbc8101de0 2024-09-19T07:32:58.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.884+0000 7ffbc7e00640 1 --2- 172.21.15.159:0/2825301108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbc8101de0 0x7ffbc819a600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8006010 tx=0x7ffbb8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:58.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.885+0000 7ffbc5600640 1 -- 172.21.15.159:0/2825301108 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb803f030 con 0x7ffbc8101de0 2024-09-19T07:32:58.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.885+0000 7ffbc5600640 1 -- 172.21.15.159:0/2825301108 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbb8004440 con 0x7ffbc8101de0 2024-09-19T07:32:58.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.885+0000 7ffbc5600640 1 -- 172.21.15.159:0/2825301108 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8035070 con 0x7ffbc8101de0 2024-09-19T07:32:58.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.885+0000 7ffbcf000640 1 -- 172.21.15.159:0/2825301108 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc819f840 con 0x7ffbc8101de0 2024-09-19T07:32:58.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.885+0000 7ffbcf000640 1 -- 172.21.15.159:0/2825301108 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc819fcd0 con 0x7ffbc8101de0 2024-09-19T07:32:58.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.886+0000 7ffbcf000640 1 -- 172.21.15.159:0/2825301108 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc81021c0 con 0x7ffbc8101de0 2024-09-19T07:32:58.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.888+0000 7ffbc5600640 1 -- 172.21.15.159:0/2825301108 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ffbb8042030 con 0x7ffbc8101de0 2024-09-19T07:32:58.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.889+0000 7ffbc5600640 1 --2- 172.21.15.159:0/2825301108 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ffbac0a3350 0x7ffbac0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:58.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.889+0000 7ffbc5600640 1 -- 172.21.15.159:0/2825301108 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7ffbb80f0220 con 0x7ffbc8101de0 2024-09-19T07:32:58.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.890+0000 7ffbbf400640 1 --2- 172.21.15.159:0/2825301108 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ffbac0a3350 0x7ffbac0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:58.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.890+0000 7ffbbf400640 1 --2- 172.21.15.159:0/2825301108 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ffbac0a3350 0x7ffbac0a5810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8076480 tx=0x7ffbb0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:58.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:58.893+0000 7ffbc5600640 1 -- 172.21.15.159:0/2825301108 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ffbb80bca90 con 0x7ffbc8101de0 2024-09-19T07:32:59.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:59.069+0000 7ffbcf000640 1 -- 172.21.15.159:0/2825301108 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffbc810de00 con 0x7ffbac0a3350 2024-09-19T07:32:59.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:59.077+0000 7ffbc5600640 1 -- 172.21.15.159:0/2825301108 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340801 (secure 0 0 0) 0x7ffbc810de00 con 0x7ffbac0a3350 2024-09-19T07:32:59.084 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:32:59.085 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-19T07:32:59.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:59.082+0000 7ffbcf000640 1 -- 172.21.15.159:0/2825301108 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ffbac0a3350 msgr2=0x7ffbac0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:59.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:59.082+0000 7ffbcf000640 1 --2- 172.21.15.159:0/2825301108 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ffbac0a3350 0x7ffbac0a5810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8076480 tx=0x7ffbb0005f90 comp rx=0 tx=0).stop 2024-09-19T07:32:59.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:59.082+0000 7ffbcf000640 1 -- 172.21.15.159:0/2825301108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbc8101de0 msgr2=0x7ffbc819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:59.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:59.082+0000 7ffbcf000640 1 --2- 172.21.15.159:0/2825301108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbc8101de0 0x7ffbc819a600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8006010 tx=0x7ffbb8004040 comp rx=0 tx=0).stop 2024-09-19T07:32:59.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:59.082+0000 7ffbcf000640 1 -- 172.21.15.159:0/2825301108 shutdown_connections 2024-09-19T07:32:59.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:59.082+0000 7ffbcf000640 1 --2- 172.21.15.159:0/2825301108 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7ffbac0a3350 0x7ffbac0a5810 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:59.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:59.082+0000 7ffbcf000640 1 --2- 172.21.15.159:0/2825301108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbc8106f10 0x7ffbc819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:59.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:59.082+0000 7ffbcf000640 1 --2- 172.21.15.159:0/2825301108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbc8102790 0x7ffbc819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:59.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:59.082+0000 7ffbcf000640 1 --2- 172.21.15.159:0/2825301108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbc8101de0 0x7ffbc819a600 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:59.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:59.082+0000 7ffbcf000640 1 -- 172.21.15.159:0/2825301108 >> 172.21.15.159:0/2825301108 conn(0x7ffbc80fd8c0 msgr2=0x7ffbc80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:59.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:59.083+0000 7ffbcf000640 1 -- 172.21.15.159:0/2825301108 shutdown_connections 2024-09-19T07:32:59.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:32:59.083+0000 7ffbcf000640 1 -- 172.21.15.159:0/2825301108 wait complete. 2024-09-19T07:32:59.889 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-19T07:32:58.719989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656521,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":539,"num_read_kb":385,"num_write":348,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":354396,"kb_used_data":13388,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124587428,"statfs":{"total":1151940427776,"available":1151577526272,"internally_reserved":0,"allocated":13709312,"data_stored":8810189,"data_compressed":501084,"data_compressed_allocated":688128,"data_compressed_original":2875392,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003025"},"pg_stats":[{"pgid":"2.1f","version":"98'9","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186519+0000","last_change":"2024-09-19T07:30:59.408103+0000","last_active":"2024-09-19T07:32:32.186519+0000","last_peered":"2024-09-19T07:32:32.186519+0000","last_clean":"2024-09-19T07:32:32.186519+0000","last_became_active":"2024-09-19T07:30:59.407585+0000","last_became_peered":"2024-09-19T07:30:59.407585+0000","last_unstale":"2024-09-19T07:32:32.186519+0000","last_undegraded":"2024-09-19T07:32:32.186519+0000","last_fullsized":"2024-09-19T07:32:32.186519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:03.104939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841762+0000","last_change":"2024-09-19T07:31:02.438518+0000","last_active":"2024-09-19T07:32:31.841762+0000","last_peered":"2024-09-19T07:32:31.841762+0000","last_clean":"2024-09-19T07:32:31.841762+0000","last_became_active":"2024-09-19T07:31:02.438025+0000","last_became_peered":"2024-09-19T07:31:02.438025+0000","last_unstale":"2024-09-19T07:32:31.841762+0000","last_undegraded":"2024-09-19T07:32:31.841762+0000","last_fullsized":"2024-09-19T07:32:31.841762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:59.148172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839630+0000","last_change":"2024-09-19T07:31:12.503517+0000","last_active":"2024-09-19T07:32:31.839630+0000","last_peered":"2024-09-19T07:32:31.839630+0000","last_clean":"2024-09-19T07:32:31.839630+0000","last_became_active":"2024-09-19T07:31:12.503155+0000","last_became_peered":"2024-09-19T07:31:12.503155+0000","last_unstale":"2024-09-19T07:32:31.839630+0000","last_undegraded":"2024-09-19T07:32:31.839630+0000","last_fullsized":"2024-09-19T07:32:31.839630+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:15.475388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842008+0000","last_change":"2024-09-19T07:30:59.445107+0000","last_active":"2024-09-19T07:32:31.842008+0000","last_peered":"2024-09-19T07:32:31.842008+0000","last_clean":"2024-09-19T07:32:31.842008+0000","last_became_active":"2024-09-19T07:30:59.444782+0000","last_became_peered":"2024-09-19T07:30:59.444782+0000","last_unstale":"2024-09-19T07:32:31.842008+0000","last_undegraded":"2024-09-19T07:32:31.842008+0000","last_fullsized":"2024-09-19T07:32:31.842008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:37.582314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186057+0000","last_change":"2024-09-19T07:31:02.437690+0000","last_active":"2024-09-19T07:32:32.186057+0000","last_peered":"2024-09-19T07:32:32.186057+0000","last_clean":"2024-09-19T07:32:32.186057+0000","last_became_active":"2024-09-19T07:31:02.429969+0000","last_became_peered":"2024-09-19T07:31:02.429969+0000","last_unstale":"2024-09-19T07:32:32.186057+0000","last_undegraded":"2024-09-19T07:32:32.186057+0000","last_fullsized":"2024-09-19T07:32:32.186057+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:23.227114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843148+0000","last_change":"2024-09-19T07:31:12.500314+0000","last_active":"2024-09-19T07:32:31.843148+0000","last_peered":"2024-09-19T07:32:31.843148+0000","last_clean":"2024-09-19T07:32:31.843148+0000","last_became_active":"2024-09-19T07:31:12.499529+0000","last_became_peered":"2024-09-19T07:31:12.499529+0000","last_unstale":"2024-09-19T07:32:31.843148+0000","last_undegraded":"2024-09-19T07:32:31.843148+0000","last_fullsized":"2024-09-19T07:32:31.843148+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:46.805336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.844209+0000","last_change":"2024-09-19T07:30:59.408974+0000","last_active":"2024-09-19T07:32:31.844209+0000","last_peered":"2024-09-19T07:32:31.844209+0000","last_clean":"2024-09-19T07:32:31.844209+0000","last_became_active":"2024-09-19T07:30:59.408606+0000","last_became_peered":"2024-09-19T07:30:59.408606+0000","last_unstale":"2024-09-19T07:32:31.844209+0000","last_undegraded":"2024-09-19T07:32:31.844209+0000","last_fullsized":"2024-09-19T07:32:31.844209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:21.962123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185445+0000","last_change":"2024-09-19T07:31:02.431202+0000","last_active":"2024-09-19T07:32:32.185445+0000","last_peered":"2024-09-19T07:32:32.185445+0000","last_clean":"2024-09-19T07:32:32.185445+0000","last_became_active":"2024-09-19T07:31:02.430789+0000","last_became_peered":"2024-09-19T07:31:02.430789+0000","last_unstale":"2024-09-19T07:32:32.185445+0000","last_undegraded":"2024-09-19T07:32:32.185445+0000","last_fullsized":"2024-09-19T07:32:32.185445+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:40.750108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843603+0000","last_change":"2024-09-19T07:31:12.500000+0000","last_active":"2024-09-19T07:32:31.843603+0000","last_peered":"2024-09-19T07:32:31.843603+0000","last_clean":"2024-09-19T07:32:31.843603+0000","last_became_active":"2024-09-19T07:31:12.499526+0000","last_became_peered":"2024-09-19T07:31:12.499526+0000","last_unstale":"2024-09-19T07:32:31.843603+0000","last_undegraded":"2024-09-19T07:32:31.843603+0000","last_fullsized":"2024-09-19T07:32:31.843603+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:15.449391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843823+0000","last_change":"2024-09-19T07:30:59.411645+0000","last_active":"2024-09-19T07:32:31.843823+0000","last_peered":"2024-09-19T07:32:31.843823+0000","last_clean":"2024-09-19T07:32:31.843823+0000","last_became_active":"2024-09-19T07:30:59.411262+0000","last_became_peered":"2024-09-19T07:30:59.411262+0000","last_unstale":"2024-09-19T07:32:31.843823+0000","last_undegraded":"2024-09-19T07:32:31.843823+0000","last_fullsized":"2024-09-19T07:32:31.843823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:53.538203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842984+0000","last_change":"2024-09-19T07:31:32.247958+0000","last_active":"2024-09-19T07:32:31.842984+0000","last_peered":"2024-09-19T07:32:31.842984+0000","last_clean":"2024-09-19T07:32:31.842984+0000","last_became_active":"2024-09-19T07:31:32.247521+0000","last_became_peered":"2024-09-19T07:31:32.247521+0000","last_unstale":"2024-09-19T07:32:31.842984+0000","last_undegraded":"2024-09-19T07:32:31.842984+0000","last_fullsized":"2024-09-19T07:32:31.842984+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:38.291915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842725+0000","last_change":"2024-09-19T07:31:12.500070+0000","last_active":"2024-09-19T07:32:31.842725+0000","last_peered":"2024-09-19T07:32:31.842725+0000","last_clean":"2024-09-19T07:32:31.842725+0000","last_became_active":"2024-09-19T07:31:12.499658+0000","last_became_peered":"2024-09-19T07:31:12.499658+0000","last_unstale":"2024-09-19T07:32:31.842725+0000","last_undegraded":"2024-09-19T07:32:31.842725+0000","last_fullsized":"2024-09-19T07:32:31.842725+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:08.955749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840047+0000","last_change":"2024-09-19T07:31:02.426972+0000","last_active":"2024-09-19T07:32:31.840047+0000","last_peered":"2024-09-19T07:32:31.840047+0000","last_clean":"2024-09-19T07:32:31.840047+0000","last_became_active":"2024-09-19T07:31:02.426661+0000","last_became_peered":"2024-09-19T07:31:02.426661+0000","last_unstale":"2024-09-19T07:32:31.840047+0000","last_undegraded":"2024-09-19T07:32:31.840047+0000","last_fullsized":"2024-09-19T07:32:31.840047+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:26.943324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842989+0000","last_change":"2024-09-19T07:30:59.410056+0000","last_active":"2024-09-19T07:32:31.842989+0000","last_peered":"2024-09-19T07:32:31.842989+0000","last_clean":"2024-09-19T07:32:31.842989+0000","last_became_active":"2024-09-19T07:30:59.409669+0000","last_became_peered":"2024-09-19T07:30:59.409669+0000","last_unstale":"2024-09-19T07:32:31.842989+0000","last_undegraded":"2024-09-19T07:32:31.842989+0000","last_fullsized":"2024-09-19T07:32:31.842989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:43.169905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842918+0000","last_change":"2024-09-19T07:31:12.557969+0000","last_active":"2024-09-19T07:32:31.842918+0000","last_peered":"2024-09-19T07:32:31.842918+0000","last_clean":"2024-09-19T07:32:31.842918+0000","last_became_active":"2024-09-19T07:31:12.556692+0000","last_became_peered":"2024-09-19T07:31:12.556692+0000","last_unstale":"2024-09-19T07:32:31.842918+0000","last_undegraded":"2024-09-19T07:32:31.842918+0000","last_fullsized":"2024-09-19T07:32:31.842918+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.563341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841908+0000","last_change":"2024-09-19T07:31:02.438979+0000","last_active":"2024-09-19T07:32:31.841908+0000","last_peered":"2024-09-19T07:32:31.841908+0000","last_clean":"2024-09-19T07:32:31.841908+0000","last_became_active":"2024-09-19T07:31:02.437192+0000","last_became_peered":"2024-09-19T07:31:02.437192+0000","last_unstale":"2024-09-19T07:32:31.841908+0000","last_undegraded":"2024-09-19T07:32:31.841908+0000","last_fullsized":"2024-09-19T07:32:31.841908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:39.957366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839831+0000","last_change":"2024-09-19T07:30:59.406970+0000","last_active":"2024-09-19T07:32:31.839831+0000","last_peered":"2024-09-19T07:32:31.839831+0000","last_clean":"2024-09-19T07:32:31.839831+0000","last_became_active":"2024-09-19T07:30:59.406537+0000","last_became_peered":"2024-09-19T07:30:59.406537+0000","last_unstale":"2024-09-19T07:32:31.839831+0000","last_undegraded":"2024-09-19T07:32:31.839831+0000","last_fullsized":"2024-09-19T07:32:31.839831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:32.969944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842829+0000","last_change":"2024-09-19T07:31:12.495709+0000","last_active":"2024-09-19T07:32:31.842829+0000","last_peered":"2024-09-19T07:32:31.842829+0000","last_clean":"2024-09-19T07:32:31.842829+0000","last_became_active":"2024-09-19T07:31:12.495201+0000","last_became_peered":"2024-09-19T07:31:12.495201+0000","last_unstale":"2024-09-19T07:32:31.842829+0000","last_undegraded":"2024-09-19T07:32:31.842829+0000","last_fullsized":"2024-09-19T07:32:31.842829+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:48.710011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839225+0000","last_change":"2024-09-19T07:31:02.431032+0000","last_active":"2024-09-19T07:32:31.839225+0000","last_peered":"2024-09-19T07:32:31.839225+0000","last_clean":"2024-09-19T07:32:31.839225+0000","last_became_active":"2024-09-19T07:31:02.430691+0000","last_became_peered":"2024-09-19T07:31:02.430691+0000","last_unstale":"2024-09-19T07:32:31.839225+0000","last_undegraded":"2024-09-19T07:32:31.839225+0000","last_fullsized":"2024-09-19T07:32:31.839225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:59.627125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841894+0000","last_change":"2024-09-19T07:30:59.410931+0000","last_active":"2024-09-19T07:32:31.841894+0000","last_peered":"2024-09-19T07:32:31.841894+0000","last_clean":"2024-09-19T07:32:31.841894+0000","last_became_active":"2024-09-19T07:30:59.410516+0000","last_became_peered":"2024-09-19T07:30:59.410516+0000","last_unstale":"2024-09-19T07:32:31.841894+0000","last_undegraded":"2024-09-19T07:32:31.841894+0000","last_fullsized":"2024-09-19T07:32:31.841894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:39.688441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.837387+0000","last_change":"2024-09-19T07:31:31.809224+0000","last_active":"2024-09-19T07:32:31.837387+0000","last_peered":"2024-09-19T07:32:31.837387+0000","last_clean":"2024-09-19T07:32:31.837387+0000","last_became_active":"2024-09-19T07:31:31.808673+0000","last_became_peered":"2024-09-19T07:31:31.808673+0000","last_unstale":"2024-09-19T07:32:31.837387+0000","last_undegraded":"2024-09-19T07:32:31.837387+0000","last_fullsized":"2024-09-19T07:32:31.837387+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:38.675403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842511+0000","last_change":"2024-09-19T07:31:02.426735+0000","last_active":"2024-09-19T07:32:31.842511+0000","last_peered":"2024-09-19T07:32:31.842511+0000","last_clean":"2024-09-19T07:32:31.842511+0000","last_became_active":"2024-09-19T07:31:02.426324+0000","last_became_peered":"2024-09-19T07:31:02.426324+0000","last_unstale":"2024-09-19T07:32:31.842511+0000","last_undegraded":"2024-09-19T07:32:31.842511+0000","last_fullsized":"2024-09-19T07:32:31.842511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:10.171191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185274+0000","last_change":"2024-09-19T07:30:59.445125+0000","last_active":"2024-09-19T07:32:32.185274+0000","last_peered":"2024-09-19T07:32:32.185274+0000","last_clean":"2024-09-19T07:32:32.185274+0000","last_became_active":"2024-09-19T07:30:59.444757+0000","last_became_peered":"2024-09-19T07:30:59.444757+0000","last_unstale":"2024-09-19T07:32:32.185274+0000","last_undegraded":"2024-09-19T07:32:32.185274+0000","last_fullsized":"2024-09-19T07:32:32.185274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:34.183389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843133+0000","last_change":"2024-09-19T07:31:12.557277+0000","last_active":"2024-09-19T07:32:31.843133+0000","last_peered":"2024-09-19T07:32:31.843133+0000","last_clean":"2024-09-19T07:32:31.843133+0000","last_became_active":"2024-09-19T07:31:12.556885+0000","last_became_peered":"2024-09-19T07:31:12.556885+0000","last_unstale":"2024-09-19T07:32:31.843133+0000","last_undegraded":"2024-09-19T07:32:31.843133+0000","last_fullsized":"2024-09-19T07:32:31.843133+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:39.960404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186040+0000","last_change":"2024-09-19T07:31:02.441712+0000","last_active":"2024-09-19T07:32:32.186040+0000","last_peered":"2024-09-19T07:32:32.186040+0000","last_clean":"2024-09-19T07:32:32.186040+0000","last_became_active":"2024-09-19T07:31:02.440039+0000","last_became_peered":"2024-09-19T07:31:02.440039+0000","last_unstale":"2024-09-19T07:32:32.186040+0000","last_undegraded":"2024-09-19T07:32:32.186040+0000","last_fullsized":"2024-09-19T07:32:32.186040+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:13.601774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843256+0000","last_change":"2024-09-19T07:31:00.027265+0000","last_active":"2024-09-19T07:32:31.843256+0000","last_peered":"2024-09-19T07:32:31.843256+0000","last_clean":"2024-09-19T07:32:31.843256+0000","last_became_active":"2024-09-19T07:31:00.026924+0000","last_became_peered":"2024-09-19T07:31:00.026924+0000","last_unstale":"2024-09-19T07:32:31.843256+0000","last_undegraded":"2024-09-19T07:32:31.843256+0000","last_fullsized":"2024-09-19T07:32:31.843256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:05.290978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839599+0000","last_change":"2024-09-19T07:31:12.495613+0000","last_active":"2024-09-19T07:32:31.839599+0000","last_peered":"2024-09-19T07:32:31.839599+0000","last_clean":"2024-09-19T07:32:31.839599+0000","last_became_active":"2024-09-19T07:31:12.495218+0000","last_became_peered":"2024-09-19T07:31:12.495218+0000","last_unstale":"2024-09-19T07:32:31.839599+0000","last_undegraded":"2024-09-19T07:32:31.839599+0000","last_fullsized":"2024-09-19T07:32:31.839599+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:27.651671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841839+0000","last_change":"2024-09-19T07:31:02.442902+0000","last_active":"2024-09-19T07:32:31.841839+0000","last_peered":"2024-09-19T07:32:31.841839+0000","last_clean":"2024-09-19T07:32:31.841839+0000","last_became_active":"2024-09-19T07:31:02.437173+0000","last_became_peered":"2024-09-19T07:31:02.437173+0000","last_unstale":"2024-09-19T07:32:31.841839+0000","last_undegraded":"2024-09-19T07:32:31.841839+0000","last_fullsized":"2024-09-19T07:32:31.841839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:40:44.760641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'10","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186477+0000","last_change":"2024-09-19T07:30:59.406488+0000","last_active":"2024-09-19T07:32:32.186477+0000","last_peered":"2024-09-19T07:32:32.186477+0000","last_clean":"2024-09-19T07:32:32.186477+0000","last_became_active":"2024-09-19T07:30:59.406121+0000","last_became_peered":"2024-09-19T07:30:59.406121+0000","last_unstale":"2024-09-19T07:32:32.186477+0000","last_undegraded":"2024-09-19T07:32:32.186477+0000","last_fullsized":"2024-09-19T07:32:32.186477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:14:50.534009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841342+0000","last_change":"2024-09-19T07:31:12.558127+0000","last_active":"2024-09-19T07:32:31.841342+0000","last_peered":"2024-09-19T07:32:31.841342+0000","last_clean":"2024-09-19T07:32:31.841342+0000","last_became_active":"2024-09-19T07:31:12.557078+0000","last_became_peered":"2024-09-19T07:31:12.557078+0000","last_unstale":"2024-09-19T07:32:31.841342+0000","last_undegraded":"2024-09-19T07:32:31.841342+0000","last_fullsized":"2024-09-19T07:32:31.841342+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:23.076116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836615+0000","last_change":"2024-09-19T07:31:02.433918+0000","last_active":"2024-09-19T07:32:31.836615+0000","last_peered":"2024-09-19T07:32:31.836615+0000","last_clean":"2024-09-19T07:32:31.836615+0000","last_became_active":"2024-09-19T07:31:02.430802+0000","last_became_peered":"2024-09-19T07:31:02.430802+0000","last_unstale":"2024-09-19T07:32:31.836615+0000","last_undegraded":"2024-09-19T07:32:31.836615+0000","last_fullsized":"2024-09-19T07:32:31.836615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:28.948441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.844215+0000","last_change":"2024-09-19T07:30:59.444178+0000","last_active":"2024-09-19T07:32:31.844215+0000","last_peered":"2024-09-19T07:32:31.844215+0000","last_clean":"2024-09-19T07:32:31.844215+0000","last_became_active":"2024-09-19T07:30:59.443847+0000","last_became_peered":"2024-09-19T07:30:59.443847+0000","last_unstale":"2024-09-19T07:32:31.844215+0000","last_undegraded":"2024-09-19T07:32:31.844215+0000","last_fullsized":"2024-09-19T07:32:31.844215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:08.588244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839855+0000","last_change":"2024-09-19T07:31:12.508847+0000","last_active":"2024-09-19T07:32:31.839855+0000","last_peered":"2024-09-19T07:32:31.839855+0000","last_clean":"2024-09-19T07:32:31.839855+0000","last_became_active":"2024-09-19T07:31:12.508371+0000","last_became_peered":"2024-09-19T07:31:12.508371+0000","last_unstale":"2024-09-19T07:32:31.839855+0000","last_undegraded":"2024-09-19T07:32:31.839855+0000","last_fullsized":"2024-09-19T07:32:31.839855+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:27.420724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.837169+0000","last_change":"2024-09-19T07:31:02.438213+0000","last_active":"2024-09-19T07:32:31.837169+0000","last_peered":"2024-09-19T07:32:31.837169+0000","last_clean":"2024-09-19T07:32:31.837169+0000","last_became_active":"2024-09-19T07:31:02.437892+0000","last_became_peered":"2024-09-19T07:31:02.437892+0000","last_unstale":"2024-09-19T07:32:31.837169+0000","last_undegraded":"2024-09-19T07:32:31.837169+0000","last_fullsized":"2024-09-19T07:32:31.837169+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:50.249962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186288+0000","last_change":"2024-09-19T07:30:59.445523+0000","last_active":"2024-09-19T07:32:32.186288+0000","last_peered":"2024-09-19T07:32:32.186288+0000","last_clean":"2024-09-19T07:32:32.186288+0000","last_became_active":"2024-09-19T07:30:59.445118+0000","last_became_peered":"2024-09-19T07:30:59.445118+0000","last_unstale":"2024-09-19T07:32:32.186288+0000","last_undegraded":"2024-09-19T07:32:32.186288+0000","last_fullsized":"2024-09-19T07:32:32.186288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:35.369102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842543+0000","last_change":"2024-09-19T07:31:12.557541+0000","last_active":"2024-09-19T07:32:31.842543+0000","last_peered":"2024-09-19T07:32:31.842543+0000","last_clean":"2024-09-19T07:32:31.842543+0000","last_became_active":"2024-09-19T07:31:12.557130+0000","last_became_peered":"2024-09-19T07:31:12.557130+0000","last_unstale":"2024-09-19T07:32:31.842543+0000","last_undegraded":"2024-09-19T07:32:31.842543+0000","last_fullsized":"2024-09-19T07:32:31.842543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:52.610222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.837091+0000","last_change":"2024-09-19T07:31:02.434743+0000","last_active":"2024-09-19T07:32:31.837091+0000","last_peered":"2024-09-19T07:32:31.837091+0000","last_clean":"2024-09-19T07:32:31.837091+0000","last_became_active":"2024-09-19T07:31:02.434312+0000","last_became_peered":"2024-09-19T07:31:02.434312+0000","last_unstale":"2024-09-19T07:32:31.837091+0000","last_undegraded":"2024-09-19T07:32:31.837091+0000","last_fullsized":"2024-09-19T07:32:31.837091+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:24:00.569195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840169+0000","last_change":"2024-09-19T07:30:59.409893+0000","last_active":"2024-09-19T07:32:31.840169+0000","last_peered":"2024-09-19T07:32:31.840169+0000","last_clean":"2024-09-19T07:32:31.840169+0000","last_became_active":"2024-09-19T07:30:59.409440+0000","last_became_peered":"2024-09-19T07:30:59.409440+0000","last_unstale":"2024-09-19T07:32:31.840169+0000","last_undegraded":"2024-09-19T07:32:31.840169+0000","last_fullsized":"2024-09-19T07:32:31.840169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:45.999596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836583+0000","last_change":"2024-09-19T07:31:12.505965+0000","last_active":"2024-09-19T07:32:31.836583+0000","last_peered":"2024-09-19T07:32:31.836583+0000","last_clean":"2024-09-19T07:32:31.836583+0000","last_became_active":"2024-09-19T07:31:12.504675+0000","last_became_peered":"2024-09-19T07:31:12.504675+0000","last_unstale":"2024-09-19T07:32:31.836583+0000","last_undegraded":"2024-09-19T07:32:31.836583+0000","last_fullsized":"2024-09-19T07:32:31.836583+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:47.345020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842290+0000","last_change":"2024-09-19T07:31:31.809488+0000","last_active":"2024-09-19T07:32:31.842290+0000","last_peered":"2024-09-19T07:32:31.842290+0000","last_clean":"2024-09-19T07:32:31.842290+0000","last_became_active":"2024-09-19T07:31:31.809107+0000","last_became_peered":"2024-09-19T07:31:31.809107+0000","last_unstale":"2024-09-19T07:32:31.842290+0000","last_undegraded":"2024-09-19T07:32:31.842290+0000","last_fullsized":"2024-09-19T07:32:31.842290+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:39.757841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":114,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:55.993060+0000","last_change":"2024-09-19T07:30:59.445001+0000","last_active":"2024-09-19T07:32:55.993060+0000","last_peered":"2024-09-19T07:32:55.993060+0000","last_clean":"2024-09-19T07:32:55.993060+0000","last_became_active":"2024-09-19T07:30:59.444590+0000","last_became_peered":"2024-09-19T07:30:59.444590+0000","last_unstale":"2024-09-19T07:32:55.993060+0000","last_undegraded":"2024-09-19T07:32:55.993060+0000","last_fullsized":"2024-09-19T07:32:55.993060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:13.934085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836988+0000","last_change":"2024-09-19T07:31:12.557622+0000","last_active":"2024-09-19T07:32:31.836988+0000","last_peered":"2024-09-19T07:32:31.836988+0000","last_clean":"2024-09-19T07:32:31.836988+0000","last_became_active":"2024-09-19T07:31:12.556934+0000","last_became_peered":"2024-09-19T07:31:12.556934+0000","last_unstale":"2024-09-19T07:32:31.836988+0000","last_undegraded":"2024-09-19T07:32:31.836988+0000","last_fullsized":"2024-09-19T07:32:31.836988+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:06:31.944819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836423+0000","last_change":"2024-09-19T07:31:02.433199+0000","last_active":"2024-09-19T07:32:31.836423+0000","last_peered":"2024-09-19T07:32:31.836423+0000","last_clean":"2024-09-19T07:32:31.836423+0000","last_became_active":"2024-09-19T07:31:02.432739+0000","last_became_peered":"2024-09-19T07:31:02.432739+0000","last_unstale":"2024-09-19T07:32:31.836423+0000","last_undegraded":"2024-09-19T07:32:31.836423+0000","last_fullsized":"2024-09-19T07:32:31.836423+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:57.392885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.849682+0000","last_change":"2024-09-19T07:32:31.849682+0000","last_active":"2024-09-19T07:32:31.849682+0000","last_peered":"2024-09-19T07:32:31.849682+0000","last_clean":"2024-09-19T07:32:31.849682+0000","last_became_active":"2024-09-19T07:32:31.846644+0000","last_became_peered":"2024-09-19T07:32:31.846644+0000","last_unstale":"2024-09-19T07:32:31.849682+0000","last_undegraded":"2024-09-19T07:32:31.849682+0000","last_fullsized":"2024-09-19T07:32:31.849682+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:14.117168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843173+0000","last_change":"2024-09-19T07:31:12.557638+0000","last_active":"2024-09-19T07:32:31.843173+0000","last_peered":"2024-09-19T07:32:31.843173+0000","last_clean":"2024-09-19T07:32:31.843173+0000","last_became_active":"2024-09-19T07:31:12.557170+0000","last_became_peered":"2024-09-19T07:31:12.557170+0000","last_unstale":"2024-09-19T07:32:31.843173+0000","last_undegraded":"2024-09-19T07:32:31.843173+0000","last_fullsized":"2024-09-19T07:32:31.843173+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:14.077038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841823+0000","last_change":"2024-09-19T07:31:02.443081+0000","last_active":"2024-09-19T07:32:31.841823+0000","last_peered":"2024-09-19T07:32:31.841823+0000","last_clean":"2024-09-19T07:32:31.841823+0000","last_became_active":"2024-09-19T07:31:02.437034+0000","last_became_peered":"2024-09-19T07:31:02.437034+0000","last_unstale":"2024-09-19T07:32:31.841823+0000","last_undegraded":"2024-09-19T07:32:31.841823+0000","last_fullsized":"2024-09-19T07:32:31.841823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:55.310844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843769+0000","last_change":"2024-09-19T07:30:59.410069+0000","last_active":"2024-09-19T07:32:31.843769+0000","last_peered":"2024-09-19T07:32:31.843769+0000","last_clean":"2024-09-19T07:32:31.843769+0000","last_became_active":"2024-09-19T07:30:59.409736+0000","last_became_peered":"2024-09-19T07:30:59.409736+0000","last_unstale":"2024-09-19T07:32:31.843769+0000","last_undegraded":"2024-09-19T07:32:31.843769+0000","last_fullsized":"2024-09-19T07:32:31.843769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:59:47.095227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":361,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840287+0000","last_change":"2024-09-19T07:29:38.994670+0000","last_active":"2024-09-19T07:32:31.840287+0000","last_peered":"2024-09-19T07:32:31.840287+0000","last_clean":"2024-09-19T07:32:31.840287+0000","last_became_active":"2024-09-19T07:29:38.994229+0000","last_became_peered":"2024-09-19T07:29:38.994229+0000","last_unstale":"2024-09-19T07:32:31.840287+0000","last_undegraded":"2024-09-19T07:32:31.840287+0000","last_fullsized":"2024-09-19T07:32:31.840287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:25:55.396577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:25:55.396577+0000","last_clean_scrub_stamp":"2024-09-19T07:25:55.396577+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:06.972254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840232+0000","last_change":"2024-09-19T07:31:12.497060+0000","last_active":"2024-09-19T07:32:31.840232+0000","last_peered":"2024-09-19T07:32:31.840232+0000","last_clean":"2024-09-19T07:32:31.840232+0000","last_became_active":"2024-09-19T07:31:12.496693+0000","last_became_peered":"2024-09-19T07:31:12.496693+0000","last_unstale":"2024-09-19T07:32:31.840232+0000","last_undegraded":"2024-09-19T07:32:31.840232+0000","last_fullsized":"2024-09-19T07:32:31.840232+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:41.879585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840496+0000","last_change":"2024-09-19T07:31:02.420732+0000","last_active":"2024-09-19T07:32:31.840496+0000","last_peered":"2024-09-19T07:32:31.840496+0000","last_clean":"2024-09-19T07:32:31.840496+0000","last_became_active":"2024-09-19T07:31:02.420394+0000","last_became_peered":"2024-09-19T07:31:02.420394+0000","last_unstale":"2024-09-19T07:32:31.840496+0000","last_undegraded":"2024-09-19T07:32:31.840496+0000","last_fullsized":"2024-09-19T07:32:31.840496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:48:36.157189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839893+0000","last_change":"2024-09-19T07:31:00.027344+0000","last_active":"2024-09-19T07:32:31.839893+0000","last_peered":"2024-09-19T07:32:31.839893+0000","last_clean":"2024-09-19T07:32:31.839893+0000","last_became_active":"2024-09-19T07:31:00.026855+0000","last_became_peered":"2024-09-19T07:31:00.026855+0000","last_unstale":"2024-09-19T07:32:31.839893+0000","last_undegraded":"2024-09-19T07:32:31.839893+0000","last_fullsized":"2024-09-19T07:32:31.839893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:12.304161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185188+0000","last_change":"2024-09-19T07:31:12.495883+0000","last_active":"2024-09-19T07:32:32.185188+0000","last_peered":"2024-09-19T07:32:32.185188+0000","last_clean":"2024-09-19T07:32:32.185188+0000","last_became_active":"2024-09-19T07:31:12.495253+0000","last_became_peered":"2024-09-19T07:31:12.495253+0000","last_unstale":"2024-09-19T07:32:32.185188+0000","last_undegraded":"2024-09-19T07:32:32.185188+0000","last_fullsized":"2024-09-19T07:32:32.185188+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:56.608977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840208+0000","last_change":"2024-09-19T07:31:02.427706+0000","last_active":"2024-09-19T07:32:31.840208+0000","last_peered":"2024-09-19T07:32:31.840208+0000","last_clean":"2024-09-19T07:32:31.840208+0000","last_became_active":"2024-09-19T07:31:02.427398+0000","last_became_peered":"2024-09-19T07:31:02.427398+0000","last_unstale":"2024-09-19T07:32:31.840208+0000","last_undegraded":"2024-09-19T07:32:31.840208+0000","last_fullsized":"2024-09-19T07:32:31.840208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:13.179472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843207+0000","last_change":"2024-09-19T07:30:59.410020+0000","last_active":"2024-09-19T07:32:31.843207+0000","last_peered":"2024-09-19T07:32:31.843207+0000","last_clean":"2024-09-19T07:32:31.843207+0000","last_became_active":"2024-09-19T07:30:59.409633+0000","last_became_peered":"2024-09-19T07:30:59.409633+0000","last_unstale":"2024-09-19T07:32:31.843207+0000","last_undegraded":"2024-09-19T07:32:31.843207+0000","last_fullsized":"2024-09-19T07:32:31.843207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:49.673952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185781+0000","last_change":"2024-09-19T07:31:12.511259+0000","last_active":"2024-09-19T07:32:32.185781+0000","last_peered":"2024-09-19T07:32:32.185781+0000","last_clean":"2024-09-19T07:32:32.185781+0000","last_became_active":"2024-09-19T07:31:12.510691+0000","last_became_peered":"2024-09-19T07:31:12.510691+0000","last_unstale":"2024-09-19T07:32:32.185781+0000","last_undegraded":"2024-09-19T07:32:32.185781+0000","last_fullsized":"2024-09-19T07:32:32.185781+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:09.388922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840526+0000","last_change":"2024-09-19T07:31:02.425922+0000","last_active":"2024-09-19T07:32:31.840526+0000","last_peered":"2024-09-19T07:32:31.840526+0000","last_clean":"2024-09-19T07:32:31.840526+0000","last_became_active":"2024-09-19T07:31:02.425565+0000","last_became_peered":"2024-09-19T07:31:02.425565+0000","last_unstale":"2024-09-19T07:32:31.840526+0000","last_undegraded":"2024-09-19T07:32:31.840526+0000","last_fullsized":"2024-09-19T07:32:31.840526+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:48.997362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836916+0000","last_change":"2024-09-19T07:31:31.813529+0000","last_active":"2024-09-19T07:32:31.836916+0000","last_peered":"2024-09-19T07:32:31.836916+0000","last_clean":"2024-09-19T07:32:31.836916+0000","last_became_active":"2024-09-19T07:31:31.809205+0000","last_became_peered":"2024-09-19T07:31:31.809205+0000","last_unstale":"2024-09-19T07:32:31.836916+0000","last_undegraded":"2024-09-19T07:32:31.836916+0000","last_fullsized":"2024-09-19T07:32:31.836916+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:26.117905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841916+0000","last_change":"2024-09-19T07:31:12.497728+0000","last_active":"2024-09-19T07:32:31.841916+0000","last_peered":"2024-09-19T07:32:31.841916+0000","last_clean":"2024-09-19T07:32:31.841916+0000","last_became_active":"2024-09-19T07:31:12.497315+0000","last_became_peered":"2024-09-19T07:31:12.497315+0000","last_unstale":"2024-09-19T07:32:31.841916+0000","last_undegraded":"2024-09-19T07:32:31.841916+0000","last_fullsized":"2024-09-19T07:32:31.841916+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:12.174147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839628+0000","last_change":"2024-09-19T07:31:02.433205+0000","last_active":"2024-09-19T07:32:31.839628+0000","last_peered":"2024-09-19T07:32:31.839628+0000","last_clean":"2024-09-19T07:32:31.839628+0000","last_became_active":"2024-09-19T07:31:02.432856+0000","last_became_peered":"2024-09-19T07:31:02.432856+0000","last_unstale":"2024-09-19T07:32:31.839628+0000","last_undegraded":"2024-09-19T07:32:31.839628+0000","last_fullsized":"2024-09-19T07:32:31.839628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:31.470420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.856388+0000","last_change":"2024-09-19T07:32:31.855914+0000","last_active":"2024-09-19T07:32:31.856388+0000","last_peered":"2024-09-19T07:32:31.856388+0000","last_clean":"2024-09-19T07:32:31.856388+0000","last_became_active":"2024-09-19T07:32:31.844841+0000","last_became_peered":"2024-09-19T07:32:31.844841+0000","last_unstale":"2024-09-19T07:32:31.856388+0000","last_undegraded":"2024-09-19T07:32:31.856388+0000","last_fullsized":"2024-09-19T07:32:31.856388+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:57.051556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841966+0000","last_change":"2024-09-19T07:31:12.500715+0000","last_active":"2024-09-19T07:32:31.841966+0000","last_peered":"2024-09-19T07:32:31.841966+0000","last_clean":"2024-09-19T07:32:31.841966+0000","last_became_active":"2024-09-19T07:31:12.500401+0000","last_became_peered":"2024-09-19T07:31:12.500401+0000","last_unstale":"2024-09-19T07:32:31.841966+0000","last_undegraded":"2024-09-19T07:32:31.841966+0000","last_fullsized":"2024-09-19T07:32:31.841966+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:59.158518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842282+0000","last_change":"2024-09-19T07:31:02.429062+0000","last_active":"2024-09-19T07:32:31.842282+0000","last_peered":"2024-09-19T07:32:31.842282+0000","last_clean":"2024-09-19T07:32:31.842282+0000","last_became_active":"2024-09-19T07:31:02.428703+0000","last_became_peered":"2024-09-19T07:31:02.428703+0000","last_unstale":"2024-09-19T07:32:31.842282+0000","last_undegraded":"2024-09-19T07:32:31.842282+0000","last_fullsized":"2024-09-19T07:32:31.842282+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:30.928698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842764+0000","last_change":"2024-09-19T07:30:59.444883+0000","last_active":"2024-09-19T07:32:31.842764+0000","last_peered":"2024-09-19T07:32:31.842764+0000","last_clean":"2024-09-19T07:32:31.842764+0000","last_became_active":"2024-09-19T07:30:59.444445+0000","last_became_peered":"2024-09-19T07:30:59.444445+0000","last_unstale":"2024-09-19T07:32:31.842764+0000","last_undegraded":"2024-09-19T07:32:31.842764+0000","last_fullsized":"2024-09-19T07:32:31.842764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:29:11.166450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836866+0000","last_change":"2024-09-19T07:31:12.508431+0000","last_active":"2024-09-19T07:32:31.836866+0000","last_peered":"2024-09-19T07:32:31.836866+0000","last_clean":"2024-09-19T07:32:31.836866+0000","last_became_active":"2024-09-19T07:31:12.507826+0000","last_became_peered":"2024-09-19T07:31:12.507826+0000","last_unstale":"2024-09-19T07:32:31.836866+0000","last_undegraded":"2024-09-19T07:32:31.836866+0000","last_fullsized":"2024-09-19T07:32:31.836866+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:59.545418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839874+0000","last_change":"2024-09-19T07:31:02.423973+0000","last_active":"2024-09-19T07:32:31.839874+0000","last_peered":"2024-09-19T07:32:31.839874+0000","last_clean":"2024-09-19T07:32:31.839874+0000","last_became_active":"2024-09-19T07:31:02.423609+0000","last_became_peered":"2024-09-19T07:31:02.423609+0000","last_unstale":"2024-09-19T07:32:31.839874+0000","last_undegraded":"2024-09-19T07:32:31.839874+0000","last_fullsized":"2024-09-19T07:32:31.839874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:23:47.693286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"98'4","reported_seq":115,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:54.992259+0000","last_change":"2024-09-19T07:30:59.412862+0000","last_active":"2024-09-19T07:32:54.992259+0000","last_peered":"2024-09-19T07:32:54.992259+0000","last_clean":"2024-09-19T07:32:54.992259+0000","last_became_active":"2024-09-19T07:30:59.412401+0000","last_became_peered":"2024-09-19T07:30:59.412401+0000","last_unstale":"2024-09-19T07:32:54.992259+0000","last_undegraded":"2024-09-19T07:32:54.992259+0000","last_fullsized":"2024-09-19T07:32:54.992259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:49.461312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186408+0000","last_change":"2024-09-19T07:31:12.507714+0000","last_active":"2024-09-19T07:32:32.186408+0000","last_peered":"2024-09-19T07:32:32.186408+0000","last_clean":"2024-09-19T07:32:32.186408+0000","last_became_active":"2024-09-19T07:31:12.506054+0000","last_became_peered":"2024-09-19T07:31:12.506054+0000","last_unstale":"2024-09-19T07:32:32.186408+0000","last_undegraded":"2024-09-19T07:32:32.186408+0000","last_fullsized":"2024-09-19T07:32:32.186408+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:18.070858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840107+0000","last_change":"2024-09-19T07:31:02.424718+0000","last_active":"2024-09-19T07:32:31.840107+0000","last_peered":"2024-09-19T07:32:31.840107+0000","last_clean":"2024-09-19T07:32:31.840107+0000","last_became_active":"2024-09-19T07:31:02.424351+0000","last_became_peered":"2024-09-19T07:31:02.424351+0000","last_unstale":"2024-09-19T07:32:31.840107+0000","last_undegraded":"2024-09-19T07:32:31.840107+0000","last_fullsized":"2024-09-19T07:32:31.840107+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:59.396752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842555+0000","last_change":"2024-09-19T07:30:59.410551+0000","last_active":"2024-09-19T07:32:31.842555+0000","last_peered":"2024-09-19T07:32:31.842555+0000","last_clean":"2024-09-19T07:32:31.842555+0000","last_became_active":"2024-09-19T07:30:59.410226+0000","last_became_peered":"2024-09-19T07:30:59.410226+0000","last_unstale":"2024-09-19T07:32:31.842555+0000","last_undegraded":"2024-09-19T07:32:31.842555+0000","last_fullsized":"2024-09-19T07:32:31.842555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:18.815424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":35,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839094+0000","last_change":"2024-09-19T07:31:32.532426+0000","last_active":"2024-09-19T07:32:31.839094+0000","last_peered":"2024-09-19T07:32:31.839094+0000","last_clean":"2024-09-19T07:32:31.839094+0000","last_became_active":"2024-09-19T07:31:32.532034+0000","last_became_peered":"2024-09-19T07:31:32.532034+0000","last_unstale":"2024-09-19T07:32:31.839094+0000","last_undegraded":"2024-09-19T07:32:31.839094+0000","last_fullsized":"2024-09-19T07:32:31.839094+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:09.421369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836541+0000","last_change":"2024-09-19T07:31:02.444240+0000","last_active":"2024-09-19T07:32:31.836541+0000","last_peered":"2024-09-19T07:32:31.836541+0000","last_clean":"2024-09-19T07:32:31.836541+0000","last_became_active":"2024-09-19T07:31:02.443808+0000","last_became_peered":"2024-09-19T07:31:02.443808+0000","last_unstale":"2024-09-19T07:32:31.836541+0000","last_undegraded":"2024-09-19T07:32:31.836541+0000","last_fullsized":"2024-09-19T07:32:31.836541+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:12.898708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.852906+0000","last_change":"2024-09-19T07:32:31.852388+0000","last_active":"2024-09-19T07:32:31.852906+0000","last_peered":"2024-09-19T07:32:31.852906+0000","last_clean":"2024-09-19T07:32:31.852906+0000","last_became_active":"2024-09-19T07:32:31.843592+0000","last_became_peered":"2024-09-19T07:32:31.843592+0000","last_unstale":"2024-09-19T07:32:31.852906+0000","last_undegraded":"2024-09-19T07:32:31.852906+0000","last_fullsized":"2024-09-19T07:32:31.852906+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:03.020981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.837202+0000","last_change":"2024-09-19T07:31:32.248185+0000","last_active":"2024-09-19T07:32:31.837202+0000","last_peered":"2024-09-19T07:32:31.837202+0000","last_clean":"2024-09-19T07:32:31.837202+0000","last_became_active":"2024-09-19T07:31:32.247514+0000","last_became_peered":"2024-09-19T07:31:32.247514+0000","last_unstale":"2024-09-19T07:32:31.837202+0000","last_undegraded":"2024-09-19T07:32:31.837202+0000","last_fullsized":"2024-09-19T07:32:31.837202+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:09.713374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.844092+0000","last_change":"2024-09-19T07:31:02.421038+0000","last_active":"2024-09-19T07:32:31.844092+0000","last_peered":"2024-09-19T07:32:31.844092+0000","last_clean":"2024-09-19T07:32:31.844092+0000","last_became_active":"2024-09-19T07:31:02.420715+0000","last_became_peered":"2024-09-19T07:31:02.420715+0000","last_unstale":"2024-09-19T07:32:31.844092+0000","last_undegraded":"2024-09-19T07:32:31.844092+0000","last_fullsized":"2024-09-19T07:32:31.844092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:39.943123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"98'31","reported_seq":209,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:54.991983+0000","last_change":"2024-09-19T07:30:59.414095+0000","last_active":"2024-09-19T07:32:54.991983+0000","last_peered":"2024-09-19T07:32:54.991983+0000","last_clean":"2024-09-19T07:32:54.991983+0000","last_became_active":"2024-09-19T07:30:59.413431+0000","last_became_peered":"2024-09-19T07:30:59.413431+0000","last_unstale":"2024-09-19T07:32:54.991983+0000","last_undegraded":"2024-09-19T07:32:54.991983+0000","last_fullsized":"2024-09-19T07:32:54.991983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:00.573369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61523,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":135,"num_read_kb":127,"num_write":32,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185929+0000","last_change":"2024-09-19T07:31:12.500563+0000","last_active":"2024-09-19T07:32:32.185929+0000","last_peered":"2024-09-19T07:32:32.185929+0000","last_clean":"2024-09-19T07:32:32.185929+0000","last_became_active":"2024-09-19T07:31:12.500170+0000","last_became_peered":"2024-09-19T07:31:12.500170+0000","last_unstale":"2024-09-19T07:32:32.185929+0000","last_undegraded":"2024-09-19T07:32:32.185929+0000","last_fullsized":"2024-09-19T07:32:32.185929+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:17.897055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185467+0000","last_change":"2024-09-19T07:31:02.431931+0000","last_active":"2024-09-19T07:32:32.185467+0000","last_peered":"2024-09-19T07:32:32.185467+0000","last_clean":"2024-09-19T07:32:32.185467+0000","last_became_active":"2024-09-19T07:31:02.431579+0000","last_became_peered":"2024-09-19T07:31:02.431579+0000","last_unstale":"2024-09-19T07:32:32.185467+0000","last_undegraded":"2024-09-19T07:32:32.185467+0000","last_fullsized":"2024-09-19T07:32:32.185467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:18.260616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843114+0000","last_change":"2024-09-19T07:30:59.445150+0000","last_active":"2024-09-19T07:32:31.843114+0000","last_peered":"2024-09-19T07:32:31.843114+0000","last_clean":"2024-09-19T07:32:31.843114+0000","last_became_active":"2024-09-19T07:30:59.444776+0000","last_became_peered":"2024-09-19T07:30:59.444776+0000","last_unstale":"2024-09-19T07:32:31.843114+0000","last_undegraded":"2024-09-19T07:32:31.843114+0000","last_fullsized":"2024-09-19T07:32:31.843114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:30.654652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843420+0000","last_change":"2024-09-19T07:31:32.248246+0000","last_active":"2024-09-19T07:32:31.843420+0000","last_peered":"2024-09-19T07:32:31.843420+0000","last_clean":"2024-09-19T07:32:31.843420+0000","last_became_active":"2024-09-19T07:31:32.247456+0000","last_became_peered":"2024-09-19T07:31:32.247456+0000","last_unstale":"2024-09-19T07:32:31.843420+0000","last_undegraded":"2024-09-19T07:32:31.843420+0000","last_fullsized":"2024-09-19T07:32:31.843420+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:31:44.320026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186054+0000","last_change":"2024-09-19T07:31:02.441924+0000","last_active":"2024-09-19T07:32:32.186054+0000","last_peered":"2024-09-19T07:32:32.186054+0000","last_clean":"2024-09-19T07:32:32.186054+0000","last_became_active":"2024-09-19T07:31:02.438247+0000","last_became_peered":"2024-09-19T07:31:02.438247+0000","last_unstale":"2024-09-19T07:32:32.186054+0000","last_undegraded":"2024-09-19T07:32:32.186054+0000","last_fullsized":"2024-09-19T07:32:32.186054+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:12.878185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840450+0000","last_change":"2024-09-19T07:30:59.409679+0000","last_active":"2024-09-19T07:32:31.840450+0000","last_peered":"2024-09-19T07:32:31.840450+0000","last_clean":"2024-09-19T07:32:31.840450+0000","last_became_active":"2024-09-19T07:30:59.409313+0000","last_became_peered":"2024-09-19T07:30:59.409313+0000","last_unstale":"2024-09-19T07:32:31.840450+0000","last_undegraded":"2024-09-19T07:32:31.840450+0000","last_fullsized":"2024-09-19T07:32:31.840450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:42.121358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185477+0000","last_change":"2024-09-19T07:31:12.506596+0000","last_active":"2024-09-19T07:32:32.185477+0000","last_peered":"2024-09-19T07:32:32.185477+0000","last_clean":"2024-09-19T07:32:32.185477+0000","last_became_active":"2024-09-19T07:31:12.505107+0000","last_became_peered":"2024-09-19T07:31:12.505107+0000","last_unstale":"2024-09-19T07:32:32.185477+0000","last_undegraded":"2024-09-19T07:32:32.185477+0000","last_fullsized":"2024-09-19T07:32:32.185477+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:39.157988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185730+0000","last_change":"2024-09-19T07:31:02.429883+0000","last_active":"2024-09-19T07:32:32.185730+0000","last_peered":"2024-09-19T07:32:32.185730+0000","last_clean":"2024-09-19T07:32:32.185730+0000","last_became_active":"2024-09-19T07:31:02.429554+0000","last_became_peered":"2024-09-19T07:31:02.429554+0000","last_unstale":"2024-09-19T07:32:32.185730+0000","last_undegraded":"2024-09-19T07:32:32.185730+0000","last_fullsized":"2024-09-19T07:32:32.185730+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:00.336506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":110,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:53.990638+0000","last_change":"2024-09-19T07:30:59.414735+0000","last_active":"2024-09-19T07:32:53.990638+0000","last_peered":"2024-09-19T07:32:53.990638+0000","last_clean":"2024-09-19T07:32:53.990638+0000","last_became_active":"2024-09-19T07:30:59.414385+0000","last_became_peered":"2024-09-19T07:30:59.414385+0000","last_unstale":"2024-09-19T07:32:53.990638+0000","last_undegraded":"2024-09-19T07:32:53.990638+0000","last_fullsized":"2024-09-19T07:32:53.990638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:43.668794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841496+0000","last_change":"2024-09-19T07:31:12.497242+0000","last_active":"2024-09-19T07:32:31.841496+0000","last_peered":"2024-09-19T07:32:31.841496+0000","last_clean":"2024-09-19T07:32:31.841496+0000","last_became_active":"2024-09-19T07:31:12.496857+0000","last_became_peered":"2024-09-19T07:31:12.496857+0000","last_unstale":"2024-09-19T07:32:31.841496+0000","last_undegraded":"2024-09-19T07:32:31.841496+0000","last_fullsized":"2024-09-19T07:32:31.841496+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:18.418594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185945+0000","last_change":"2024-09-19T07:31:12.504507+0000","last_active":"2024-09-19T07:32:32.185945+0000","last_peered":"2024-09-19T07:32:32.185945+0000","last_clean":"2024-09-19T07:32:32.185945+0000","last_became_active":"2024-09-19T07:31:12.503289+0000","last_became_peered":"2024-09-19T07:31:12.503289+0000","last_unstale":"2024-09-19T07:32:32.185945+0000","last_undegraded":"2024-09-19T07:32:32.185945+0000","last_fullsized":"2024-09-19T07:32:32.185945+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:21.771692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185564+0000","last_change":"2024-09-19T07:31:02.429634+0000","last_active":"2024-09-19T07:32:32.185564+0000","last_peered":"2024-09-19T07:32:32.185564+0000","last_clean":"2024-09-19T07:32:32.185564+0000","last_became_active":"2024-09-19T07:31:02.429280+0000","last_became_peered":"2024-09-19T07:31:02.429280+0000","last_unstale":"2024-09-19T07:32:32.185564+0000","last_undegraded":"2024-09-19T07:32:32.185564+0000","last_fullsized":"2024-09-19T07:32:32.185564+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:13.872594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842927+0000","last_change":"2024-09-19T07:30:59.409961+0000","last_active":"2024-09-19T07:32:31.842927+0000","last_peered":"2024-09-19T07:32:31.842927+0000","last_clean":"2024-09-19T07:32:31.842927+0000","last_became_active":"2024-09-19T07:30:59.409582+0000","last_became_peered":"2024-09-19T07:30:59.409582+0000","last_unstale":"2024-09-19T07:32:31.842927+0000","last_undegraded":"2024-09-19T07:32:31.842927+0000","last_fullsized":"2024-09-19T07:32:31.842927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:31.075378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839571+0000","last_change":"2024-09-19T07:31:12.510999+0000","last_active":"2024-09-19T07:32:31.839571+0000","last_peered":"2024-09-19T07:32:31.839571+0000","last_clean":"2024-09-19T07:32:31.839571+0000","last_became_active":"2024-09-19T07:31:12.510604+0000","last_became_peered":"2024-09-19T07:31:12.510604+0000","last_unstale":"2024-09-19T07:32:31.839571+0000","last_undegraded":"2024-09-19T07:32:31.839571+0000","last_fullsized":"2024-09-19T07:32:31.839571+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:04.779289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841488+0000","last_change":"2024-09-19T07:31:02.428108+0000","last_active":"2024-09-19T07:32:31.841488+0000","last_peered":"2024-09-19T07:32:31.841488+0000","last_clean":"2024-09-19T07:32:31.841488+0000","last_became_active":"2024-09-19T07:31:02.425537+0000","last_became_peered":"2024-09-19T07:31:02.425537+0000","last_unstale":"2024-09-19T07:32:31.841488+0000","last_undegraded":"2024-09-19T07:32:31.841488+0000","last_fullsized":"2024-09-19T07:32:31.841488+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:36.094089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836205+0000","last_change":"2024-09-19T07:30:59.404687+0000","last_active":"2024-09-19T07:32:31.836205+0000","last_peered":"2024-09-19T07:32:31.836205+0000","last_clean":"2024-09-19T07:32:31.836205+0000","last_became_active":"2024-09-19T07:30:59.404222+0000","last_became_peered":"2024-09-19T07:30:59.404222+0000","last_unstale":"2024-09-19T07:32:31.836205+0000","last_undegraded":"2024-09-19T07:32:31.836205+0000","last_fullsized":"2024-09-19T07:32:31.836205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:11.287464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.837093+0000","last_change":"2024-09-19T07:30:59.414711+0000","last_active":"2024-09-19T07:32:31.837093+0000","last_peered":"2024-09-19T07:32:31.837093+0000","last_clean":"2024-09-19T07:32:31.837093+0000","last_became_active":"2024-09-19T07:30:59.414349+0000","last_became_peered":"2024-09-19T07:30:59.414349+0000","last_unstale":"2024-09-19T07:32:31.837093+0000","last_undegraded":"2024-09-19T07:32:31.837093+0000","last_fullsized":"2024-09-19T07:32:31.837093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:57:09.910328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186040+0000","last_change":"2024-09-19T07:31:31.810186+0000","last_active":"2024-09-19T07:32:32.186040+0000","last_peered":"2024-09-19T07:32:32.186040+0000","last_clean":"2024-09-19T07:32:32.186040+0000","last_became_active":"2024-09-19T07:31:31.809861+0000","last_became_peered":"2024-09-19T07:31:31.809861+0000","last_unstale":"2024-09-19T07:32:32.186040+0000","last_undegraded":"2024-09-19T07:32:32.186040+0000","last_fullsized":"2024-09-19T07:32:32.186040+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:49.443012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839492+0000","last_change":"2024-09-19T07:31:12.506060+0000","last_active":"2024-09-19T07:32:31.839492+0000","last_peered":"2024-09-19T07:32:31.839492+0000","last_clean":"2024-09-19T07:32:31.839492+0000","last_became_active":"2024-09-19T07:31:12.505535+0000","last_became_peered":"2024-09-19T07:31:12.505535+0000","last_unstale":"2024-09-19T07:32:31.839492+0000","last_undegraded":"2024-09-19T07:32:31.839492+0000","last_fullsized":"2024-09-19T07:32:31.839492+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:02.820475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'6","reported_seq":116,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:53.991279+0000","last_change":"2024-09-19T07:30:59.408594+0000","last_active":"2024-09-19T07:32:53.991279+0000","last_peered":"2024-09-19T07:32:53.991279+0000","last_clean":"2024-09-19T07:32:53.991279+0000","last_became_active":"2024-09-19T07:30:59.408164+0000","last_became_peered":"2024-09-19T07:30:59.408164+0000","last_unstale":"2024-09-19T07:32:53.991279+0000","last_undegraded":"2024-09-19T07:32:53.991279+0000","last_fullsized":"2024-09-19T07:32:53.991279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:23.950637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843141+0000","last_change":"2024-09-19T07:31:02.419940+0000","last_active":"2024-09-19T07:32:31.843141+0000","last_peered":"2024-09-19T07:32:31.843141+0000","last_clean":"2024-09-19T07:32:31.843141+0000","last_became_active":"2024-09-19T07:31:02.419618+0000","last_became_peered":"2024-09-19T07:31:02.419618+0000","last_unstale":"2024-09-19T07:32:31.843141+0000","last_undegraded":"2024-09-19T07:32:31.843141+0000","last_fullsized":"2024-09-19T07:32:31.843141+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:18.208200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840292+0000","last_change":"2024-09-19T07:31:12.497271+0000","last_active":"2024-09-19T07:32:31.840292+0000","last_peered":"2024-09-19T07:32:31.840292+0000","last_clean":"2024-09-19T07:32:31.840292+0000","last_became_active":"2024-09-19T07:31:12.496882+0000","last_became_peered":"2024-09-19T07:31:12.496882+0000","last_unstale":"2024-09-19T07:32:31.840292+0000","last_undegraded":"2024-09-19T07:32:31.840292+0000","last_fullsized":"2024-09-19T07:32:31.840292+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:48.714941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839009+0000","last_change":"2024-09-19T07:31:02.429917+0000","last_active":"2024-09-19T07:32:31.839009+0000","last_peered":"2024-09-19T07:32:31.839009+0000","last_clean":"2024-09-19T07:32:31.839009+0000","last_became_active":"2024-09-19T07:31:02.429535+0000","last_became_peered":"2024-09-19T07:31:02.429535+0000","last_unstale":"2024-09-19T07:32:31.839009+0000","last_undegraded":"2024-09-19T07:32:31.839009+0000","last_fullsized":"2024-09-19T07:32:31.839009+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:21.958491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839931+0000","last_change":"2024-09-19T07:31:02.433135+0000","last_active":"2024-09-19T07:32:31.839931+0000","last_peered":"2024-09-19T07:32:31.839931+0000","last_clean":"2024-09-19T07:32:31.839931+0000","last_became_active":"2024-09-19T07:31:02.432800+0000","last_became_peered":"2024-09-19T07:31:02.432800+0000","last_unstale":"2024-09-19T07:32:31.839931+0000","last_undegraded":"2024-09-19T07:32:31.839931+0000","last_fullsized":"2024-09-19T07:32:31.839931+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:01.559565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839603+0000","last_change":"2024-09-19T07:31:02.438670+0000","last_active":"2024-09-19T07:32:31.839603+0000","last_peered":"2024-09-19T07:32:31.839603+0000","last_clean":"2024-09-19T07:32:31.839603+0000","last_became_active":"2024-09-19T07:31:02.438191+0000","last_became_peered":"2024-09-19T07:31:02.438191+0000","last_unstale":"2024-09-19T07:32:31.839603+0000","last_undegraded":"2024-09-19T07:32:31.839603+0000","last_fullsized":"2024-09-19T07:32:31.839603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:55.317996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185820+0000","last_change":"2024-09-19T07:31:02.434826+0000","last_active":"2024-09-19T07:32:32.185820+0000","last_peered":"2024-09-19T07:32:32.185820+0000","last_clean":"2024-09-19T07:32:32.185820+0000","last_became_active":"2024-09-19T07:31:02.433695+0000","last_became_peered":"2024-09-19T07:31:02.433695+0000","last_unstale":"2024-09-19T07:32:32.185820+0000","last_undegraded":"2024-09-19T07:32:32.185820+0000","last_fullsized":"2024-09-19T07:32:32.185820+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:33:28.866345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836527+0000","last_change":"2024-09-19T07:31:32.021091+0000","last_active":"2024-09-19T07:32:31.836527+0000","last_peered":"2024-09-19T07:32:31.836527+0000","last_clean":"2024-09-19T07:32:31.836527+0000","last_became_active":"2024-09-19T07:31:32.020537+0000","last_became_peered":"2024-09-19T07:31:32.020537+0000","last_unstale":"2024-09-19T07:32:31.836527+0000","last_undegraded":"2024-09-19T07:32:31.836527+0000","last_fullsized":"2024-09-19T07:32:31.836527+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:31.649790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839411+0000","last_change":"2024-09-19T07:31:02.424514+0000","last_active":"2024-09-19T07:32:31.839411+0000","last_peered":"2024-09-19T07:32:31.839411+0000","last_clean":"2024-09-19T07:32:31.839411+0000","last_became_active":"2024-09-19T07:31:02.424194+0000","last_became_peered":"2024-09-19T07:31:02.424194+0000","last_unstale":"2024-09-19T07:32:31.839411+0000","last_undegraded":"2024-09-19T07:32:31.839411+0000","last_fullsized":"2024-09-19T07:32:31.839411+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:41:47.039372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841312+0000","last_change":"2024-09-19T07:31:02.429624+0000","last_active":"2024-09-19T07:32:31.841312+0000","last_peered":"2024-09-19T07:32:31.841312+0000","last_clean":"2024-09-19T07:32:31.841312+0000","last_became_active":"2024-09-19T07:31:02.429316+0000","last_became_peered":"2024-09-19T07:31:02.429316+0000","last_unstale":"2024-09-19T07:32:31.841312+0000","last_undegraded":"2024-09-19T07:32:31.841312+0000","last_fullsized":"2024-09-19T07:32:31.841312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:15.736433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186389+0000","last_change":"2024-09-19T07:31:31.810511+0000","last_active":"2024-09-19T07:32:32.186389+0000","last_peered":"2024-09-19T07:32:32.186389+0000","last_clean":"2024-09-19T07:32:32.186389+0000","last_became_active":"2024-09-19T07:31:31.809805+0000","last_became_peered":"2024-09-19T07:31:31.809805+0000","last_unstale":"2024-09-19T07:32:32.186389+0000","last_undegraded":"2024-09-19T07:32:32.186389+0000","last_fullsized":"2024-09-19T07:32:32.186389+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:48.457693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842170+0000","last_change":"2024-09-19T07:31:02.433859+0000","last_active":"2024-09-19T07:32:31.842170+0000","last_peered":"2024-09-19T07:32:31.842170+0000","last_clean":"2024-09-19T07:32:31.842170+0000","last_became_active":"2024-09-19T07:31:02.433547+0000","last_became_peered":"2024-09-19T07:31:02.433547+0000","last_unstale":"2024-09-19T07:32:31.842170+0000","last_undegraded":"2024-09-19T07:32:31.842170+0000","last_fullsized":"2024-09-19T07:32:31.842170+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:09.623326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841702+0000","last_change":"2024-09-19T07:31:02.442174+0000","last_active":"2024-09-19T07:32:31.841702+0000","last_peered":"2024-09-19T07:32:31.841702+0000","last_clean":"2024-09-19T07:32:31.841702+0000","last_became_active":"2024-09-19T07:31:02.441851+0000","last_became_peered":"2024-09-19T07:31:02.441851+0000","last_unstale":"2024-09-19T07:32:31.841702+0000","last_undegraded":"2024-09-19T07:32:31.841702+0000","last_fullsized":"2024-09-19T07:32:31.841702+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:59.235498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185362+0000","last_change":"2024-09-19T07:31:02.439526+0000","last_active":"2024-09-19T07:32:32.185362+0000","last_peered":"2024-09-19T07:32:32.185362+0000","last_clean":"2024-09-19T07:32:32.185362+0000","last_became_active":"2024-09-19T07:31:02.438996+0000","last_became_peered":"2024-09-19T07:31:02.438996+0000","last_unstale":"2024-09-19T07:32:32.185362+0000","last_undegraded":"2024-09-19T07:32:32.185362+0000","last_fullsized":"2024-09-19T07:32:32.185362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:55.909727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836389+0000","last_change":"2024-09-19T07:31:02.427770+0000","last_active":"2024-09-19T07:32:31.836389+0000","last_peered":"2024-09-19T07:32:31.836389+0000","last_clean":"2024-09-19T07:32:31.836389+0000","last_became_active":"2024-09-19T07:31:02.427208+0000","last_became_peered":"2024-09-19T07:31:02.427208+0000","last_unstale":"2024-09-19T07:32:31.836389+0000","last_undegraded":"2024-09-19T07:32:31.836389+0000","last_fullsized":"2024-09-19T07:32:31.836389+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:01.171781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839160+0000","last_change":"2024-09-19T07:31:02.433089+0000","last_active":"2024-09-19T07:32:31.839160+0000","last_peered":"2024-09-19T07:32:31.839160+0000","last_clean":"2024-09-19T07:32:31.839160+0000","last_became_active":"2024-09-19T07:31:02.432744+0000","last_became_peered":"2024-09-19T07:31:02.432744+0000","last_unstale":"2024-09-19T07:32:31.839160+0000","last_undegraded":"2024-09-19T07:32:31.839160+0000","last_fullsized":"2024-09-19T07:32:31.839160+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:32.959555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840486+0000","last_change":"2024-09-19T07:31:02.435482+0000","last_active":"2024-09-19T07:32:31.840486+0000","last_peered":"2024-09-19T07:32:31.840486+0000","last_clean":"2024-09-19T07:32:31.840486+0000","last_became_active":"2024-09-19T07:31:02.435164+0000","last_became_peered":"2024-09-19T07:31:02.435164+0000","last_unstale":"2024-09-19T07:32:31.840486+0000","last_undegraded":"2024-09-19T07:32:31.840486+0000","last_fullsized":"2024-09-19T07:32:31.840486+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:54.478049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186419+0000","last_change":"2024-09-19T07:31:02.426768+0000","last_active":"2024-09-19T07:32:32.186419+0000","last_peered":"2024-09-19T07:32:32.186419+0000","last_clean":"2024-09-19T07:32:32.186419+0000","last_became_active":"2024-09-19T07:31:02.425592+0000","last_became_peered":"2024-09-19T07:31:02.425592+0000","last_unstale":"2024-09-19T07:32:32.186419+0000","last_undegraded":"2024-09-19T07:32:32.186419+0000","last_fullsized":"2024-09-19T07:32:32.186419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:44.985532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840446+0000","last_change":"2024-09-19T07:31:02.426233+0000","last_active":"2024-09-19T07:32:31.840446+0000","last_peered":"2024-09-19T07:32:31.840446+0000","last_clean":"2024-09-19T07:32:31.840446+0000","last_became_active":"2024-09-19T07:31:02.425777+0000","last_became_peered":"2024-09-19T07:31:02.425777+0000","last_unstale":"2024-09-19T07:32:31.840446+0000","last_undegraded":"2024-09-19T07:32:31.840446+0000","last_fullsized":"2024-09-19T07:32:31.840446+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:31.497641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185109+0000","last_change":"2024-09-19T07:31:02.428671+0000","last_active":"2024-09-19T07:32:32.185109+0000","last_peered":"2024-09-19T07:32:32.185109+0000","last_clean":"2024-09-19T07:32:32.185109+0000","last_became_active":"2024-09-19T07:31:02.425930+0000","last_became_peered":"2024-09-19T07:31:02.425930+0000","last_unstale":"2024-09-19T07:32:32.185109+0000","last_undegraded":"2024-09-19T07:32:32.185109+0000","last_fullsized":"2024-09-19T07:32:32.185109+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:21.721694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185416+0000","last_change":"2024-09-19T07:31:02.441450+0000","last_active":"2024-09-19T07:32:32.185416+0000","last_peered":"2024-09-19T07:32:32.185416+0000","last_clean":"2024-09-19T07:32:32.185416+0000","last_became_active":"2024-09-19T07:31:02.436673+0000","last_became_peered":"2024-09-19T07:31:02.436673+0000","last_unstale":"2024-09-19T07:32:32.185416+0000","last_undegraded":"2024-09-19T07:32:32.185416+0000","last_fullsized":"2024-09-19T07:32:32.185416+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:29.744665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185613+0000","last_change":"2024-09-19T07:31:02.427449+0000","last_active":"2024-09-19T07:32:32.185613+0000","last_peered":"2024-09-19T07:32:32.185613+0000","last_clean":"2024-09-19T07:32:32.185613+0000","last_became_active":"2024-09-19T07:31:02.426601+0000","last_became_peered":"2024-09-19T07:31:02.426601+0000","last_unstale":"2024-09-19T07:32:32.185613+0000","last_undegraded":"2024-09-19T07:32:32.185613+0000","last_fullsized":"2024-09-19T07:32:32.185613+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:36.752529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836388+0000","last_change":"2024-09-19T07:31:02.424816+0000","last_active":"2024-09-19T07:32:31.836388+0000","last_peered":"2024-09-19T07:32:31.836388+0000","last_clean":"2024-09-19T07:32:31.836388+0000","last_became_active":"2024-09-19T07:31:02.423729+0000","last_became_peered":"2024-09-19T07:31:02.423729+0000","last_unstale":"2024-09-19T07:32:31.836388+0000","last_undegraded":"2024-09-19T07:32:31.836388+0000","last_fullsized":"2024-09-19T07:32:31.836388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:39.546052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843223+0000","last_change":"2024-09-19T07:31:02.428444+0000","last_active":"2024-09-19T07:32:31.843223+0000","last_peered":"2024-09-19T07:32:31.843223+0000","last_clean":"2024-09-19T07:32:31.843223+0000","last_became_active":"2024-09-19T07:31:02.428121+0000","last_became_peered":"2024-09-19T07:31:02.428121+0000","last_unstale":"2024-09-19T07:32:31.843223+0000","last_undegraded":"2024-09-19T07:32:31.843223+0000","last_fullsized":"2024-09-19T07:32:31.843223+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:52.649412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840453+0000","last_change":"2024-09-19T07:31:02.423102+0000","last_active":"2024-09-19T07:32:31.840453+0000","last_peered":"2024-09-19T07:32:31.840453+0000","last_clean":"2024-09-19T07:32:31.840453+0000","last_became_active":"2024-09-19T07:31:02.422672+0000","last_became_peered":"2024-09-19T07:31:02.422672+0000","last_unstale":"2024-09-19T07:32:31.840453+0000","last_undegraded":"2024-09-19T07:32:31.840453+0000","last_fullsized":"2024-09-19T07:32:31.840453+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:45.733320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.837494+0000","last_change":"2024-09-19T07:31:02.434599+0000","last_active":"2024-09-19T07:32:31.837494+0000","last_peered":"2024-09-19T07:32:31.837494+0000","last_clean":"2024-09-19T07:32:31.837494+0000","last_became_active":"2024-09-19T07:31:02.434263+0000","last_became_peered":"2024-09-19T07:31:02.434263+0000","last_unstale":"2024-09-19T07:32:31.837494+0000","last_undegraded":"2024-09-19T07:32:31.837494+0000","last_fullsized":"2024-09-19T07:32:31.837494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:16.104483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185751+0000","last_change":"2024-09-19T07:31:02.429278+0000","last_active":"2024-09-19T07:32:32.185751+0000","last_peered":"2024-09-19T07:32:32.185751+0000","last_clean":"2024-09-19T07:32:32.185751+0000","last_became_active":"2024-09-19T07:31:02.428991+0000","last_became_peered":"2024-09-19T07:31:02.428991+0000","last_unstale":"2024-09-19T07:32:32.185751+0000","last_undegraded":"2024-09-19T07:32:32.185751+0000","last_fullsized":"2024-09-19T07:32:32.185751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:02.421684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841490+0000","last_change":"2024-09-19T07:31:02.436412+0000","last_active":"2024-09-19T07:32:31.841490+0000","last_peered":"2024-09-19T07:32:31.841490+0000","last_clean":"2024-09-19T07:32:31.841490+0000","last_became_active":"2024-09-19T07:31:02.436101+0000","last_became_peered":"2024-09-19T07:31:02.436101+0000","last_unstale":"2024-09-19T07:32:31.841490+0000","last_undegraded":"2024-09-19T07:32:31.841490+0000","last_fullsized":"2024-09-19T07:32:31.841490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:34.827734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839956+0000","last_change":"2024-09-19T07:31:02.423414+0000","last_active":"2024-09-19T07:32:31.839956+0000","last_peered":"2024-09-19T07:32:31.839956+0000","last_clean":"2024-09-19T07:32:31.839956+0000","last_became_active":"2024-09-19T07:31:02.423100+0000","last_became_peered":"2024-09-19T07:31:02.423100+0000","last_unstale":"2024-09-19T07:32:31.839956+0000","last_undegraded":"2024-09-19T07:32:31.839956+0000","last_fullsized":"2024-09-19T07:32:31.839956+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:15.269095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.837049+0000","last_change":"2024-09-19T07:31:02.434953+0000","last_active":"2024-09-19T07:32:31.837049+0000","last_peered":"2024-09-19T07:32:31.837049+0000","last_clean":"2024-09-19T07:32:31.837049+0000","last_became_active":"2024-09-19T07:31:02.434662+0000","last_became_peered":"2024-09-19T07:31:02.434662+0000","last_unstale":"2024-09-19T07:32:31.837049+0000","last_undegraded":"2024-09-19T07:32:31.837049+0000","last_fullsized":"2024-09-19T07:32:31.837049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:17.132108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842883+0000","last_change":"2024-09-19T07:31:02.427959+0000","last_active":"2024-09-19T07:32:31.842883+0000","last_peered":"2024-09-19T07:32:31.842883+0000","last_clean":"2024-09-19T07:32:31.842883+0000","last_became_active":"2024-09-19T07:31:02.427671+0000","last_became_peered":"2024-09-19T07:31:02.427671+0000","last_unstale":"2024-09-19T07:32:31.842883+0000","last_undegraded":"2024-09-19T07:32:31.842883+0000","last_fullsized":"2024-09-19T07:32:31.842883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:19.608005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843150+0000","last_change":"2024-09-19T07:31:02.433310+0000","last_active":"2024-09-19T07:32:31.843150+0000","last_peered":"2024-09-19T07:32:31.843150+0000","last_clean":"2024-09-19T07:32:31.843150+0000","last_became_active":"2024-09-19T07:31:02.432922+0000","last_became_peered":"2024-09-19T07:31:02.432922+0000","last_unstale":"2024-09-19T07:32:31.843150+0000","last_undegraded":"2024-09-19T07:32:31.843150+0000","last_fullsized":"2024-09-19T07:32:31.843150+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:30.043075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185720+0000","last_change":"2024-09-19T07:31:02.429452+0000","last_active":"2024-09-19T07:32:32.185720+0000","last_peered":"2024-09-19T07:32:32.185720+0000","last_clean":"2024-09-19T07:32:32.185720+0000","last_became_active":"2024-09-19T07:31:02.429142+0000","last_became_peered":"2024-09-19T07:31:02.429142+0000","last_unstale":"2024-09-19T07:32:32.185720+0000","last_undegraded":"2024-09-19T07:32:32.185720+0000","last_fullsized":"2024-09-19T07:32:32.185720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:42.107179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186005+0000","last_change":"2024-09-19T07:31:02.431197+0000","last_active":"2024-09-19T07:32:32.186005+0000","last_peered":"2024-09-19T07:32:32.186005+0000","last_clean":"2024-09-19T07:32:32.186005+0000","last_became_active":"2024-09-19T07:31:02.430812+0000","last_became_peered":"2024-09-19T07:31:02.430812+0000","last_unstale":"2024-09-19T07:32:32.186005+0000","last_undegraded":"2024-09-19T07:32:32.186005+0000","last_fullsized":"2024-09-19T07:32:32.186005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:10.682008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840341+0000","last_change":"2024-09-19T07:31:02.426753+0000","last_active":"2024-09-19T07:32:31.840341+0000","last_peered":"2024-09-19T07:32:31.840341+0000","last_clean":"2024-09-19T07:32:31.840341+0000","last_became_active":"2024-09-19T07:31:02.425870+0000","last_became_peered":"2024-09-19T07:31:02.425870+0000","last_unstale":"2024-09-19T07:32:31.840341+0000","last_undegraded":"2024-09-19T07:32:31.840341+0000","last_fullsized":"2024-09-19T07:32:31.840341+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:05.410032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65919,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":413,"num_read_kb":276,"num_write":165,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":197757,"data_compressed":5616,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247636,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1110016,"data_stored":572114,"data_compressed":32327,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579947,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28240,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716912,"statfs":{"total":95995035648,"available":95966117888,"internally_reserved":0,"allocated":1241088,"data_stored":1221242,"data_compressed":70047,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1122304,"data_stored":576390,"data_compressed":32327,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005110,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1114112,"data_stored":572192,"data_compressed":32327,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299452,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1167360,"data_stored":1168518,"data_compressed":68175,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462977,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1097728,"data_stored":567470,"data_compressed":32327,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28176,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716976,"statfs":{"total":95995035648,"available":95966183424,"internally_reserved":0,"allocated":1175552,"data_stored":1163713,"data_compressed":68175,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888140,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1110016,"data_stored":572513,"data_compressed":32327,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149777,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1110016,"data_stored":567664,"data_compressed":32327,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28184,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716968,"statfs":{"total":95995035648,"available":95966175232,"internally_reserved":0,"allocated":1179648,"data_stored":630612,"data_compressed":34199,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1159168,"data_stored":628903,"data_compressed":34199,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32288,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712864,"statfs":{"total":95995035648,"available":95961972736,"internally_reserved":0,"allocated":1122304,"data_stored":568858,"data_compressed":32327,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61523,"data_compressed":1872,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63232,"data_compressed":1872,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63494,"data_compressed":1872,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:32:59.895 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T07:33:00.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:32:59 smithi159 ceph-mon[30472]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:33:00.267 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:00.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:32:59 smithi203 ceph-mon[35335]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:33:00.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:32:59 smithi189 ceph-mon[34851]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:33:00.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.898+0000 7f4312800640 1 -- 172.21.15.159:0/3689320908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f430c108f60 msgr2=0x7f430c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:00.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.898+0000 7f4312800640 1 --2- 172.21.15.159:0/3689320908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f430c108f60 0x7f430c110ae0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4304009ca0 tx=0x7f430402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:33:00.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.898+0000 7f4312800640 1 -- 172.21.15.159:0/3689320908 shutdown_connections 2024-09-19T07:33:00.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.898+0000 7f4312800640 1 --2- 172.21.15.159:0/3689320908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f430c108f60 0x7f430c110ae0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:00.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.898+0000 7f4312800640 1 --2- 172.21.15.159:0/3689320908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f430c1047e0 0x7f430c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:00.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.898+0000 7f4312800640 1 --2- 172.21.15.159:0/3689320908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f430c103e30 0x7f430c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:00.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.898+0000 7f4312800640 1 -- 172.21.15.159:0/3689320908 >> 172.21.15.159:0/3689320908 conn(0x7f430c0ff910 msgr2=0x7f430c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:00.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.898+0000 7f4312800640 1 -- 172.21.15.159:0/3689320908 shutdown_connections 2024-09-19T07:33:00.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.898+0000 7f4312800640 1 -- 172.21.15.159:0/3689320908 wait complete. 2024-09-19T07:33:00.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.899+0000 7f4312800640 1 Processor -- start 2024-09-19T07:33:00.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.899+0000 7f4312800640 1 -- start start 2024-09-19T07:33:00.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.899+0000 7f4312800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f430c103e30 0x7f430c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:00.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.899+0000 7f4312800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f430c1047e0 0x7f430c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:00.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.899+0000 7f4312800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f430c108f60 0x7f430c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:00.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.900+0000 7f4312800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f430c113fc0 con 0x7f430c1047e0 2024-09-19T07:33:00.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.900+0000 7f4312800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f430c113e40 con 0x7f430c108f60 2024-09-19T07:33:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.900+0000 7f4312800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f430c114140 con 0x7f430c103e30 2024-09-19T07:33:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.900+0000 7f4310a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f430c1047e0 0x7f430c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.900+0000 7f4310a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f430c1047e0 0x7f430c19a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44700/0 (socket says 172.21.15.159:44700) 2024-09-19T07:33:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.900+0000 7f4310a00640 1 -- 172.21.15.159:0/125743756 learned_addr learned my addr 172.21.15.159:0/125743756 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.900+0000 7f4311e00640 1 --2- 172.21.15.159:0/125743756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f430c108f60 0x7f430c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.900+0000 7f4311400640 1 --2- 172.21.15.159:0/125743756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f430c103e30 0x7f430c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.900+0000 7f4310a00640 1 -- 172.21.15.159:0/125743756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f430c103e30 msgr2=0x7f430c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.900+0000 7f4310a00640 1 --2- 172.21.15.159:0/125743756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f430c103e30 0x7f430c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.900+0000 7f4310a00640 1 -- 172.21.15.159:0/125743756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f430c108f60 msgr2=0x7f430c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.900+0000 7f4310a00640 1 --2- 172.21.15.159:0/125743756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f430c108f60 0x7f430c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.900+0000 7f4310a00640 1 -- 172.21.15.159:0/125743756 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430c19f430 con 0x7f430c1047e0 2024-09-19T07:33:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.900+0000 7f4311e00640 1 --2- 172.21.15.159:0/125743756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f430c108f60 0x7f430c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.901+0000 7f4310a00640 1 --2- 172.21.15.159:0/125743756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f430c1047e0 0x7f430c19a9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f42f800e9f0 tx=0x7f42f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.901+0000 7f430a000640 1 -- 172.21.15.159:0/125743756 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f800cd80 con 0x7f430c1047e0 2024-09-19T07:33:00.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.901+0000 7f430a000640 1 -- 172.21.15.159:0/125743756 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f8010910 con 0x7f430c1047e0 2024-09-19T07:33:00.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.901+0000 7f430a000640 1 -- 172.21.15.159:0/125743756 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8018990 con 0x7f430c1047e0 2024-09-19T07:33:00.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.901+0000 7f4312800640 1 -- 172.21.15.159:0/125743756 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f430c19f720 con 0x7f430c1047e0 2024-09-19T07:33:00.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.901+0000 7f4312800640 1 -- 172.21.15.159:0/125743756 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f430c1a7000 con 0x7f430c1047e0 2024-09-19T07:33:00.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.902+0000 7f4312800640 1 -- 172.21.15.159:0/125743756 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f430c105310 con 0x7f430c1047e0 2024-09-19T07:33:00.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.904+0000 7f430a000640 1 -- 172.21.15.159:0/125743756 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f42f8010ac0 con 0x7f430c1047e0 2024-09-19T07:33:00.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.906+0000 7f430a000640 1 --2- 172.21.15.159:0/125743756 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f42f40a32d0 0x7f42f40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:00.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.906+0000 7f430a000640 1 -- 172.21.15.159:0/125743756 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f42f80d0140 con 0x7f430c1047e0 2024-09-19T07:33:00.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.906+0000 7f4311400640 1 --2- 172.21.15.159:0/125743756 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f42f40a32d0 0x7f42f40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:00.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.907+0000 7f4311400640 1 --2- 172.21.15.159:0/125743756 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f42f40a32d0 0x7f42f40a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4300009bd0 tx=0x7f4300005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:00.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:00.910+0000 7f430a000640 1 -- 172.21.15.159:0/125743756 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f42f809c2a0 con 0x7f430c1047e0 2024-09-19T07:33:01.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:01.082+0000 7f4312800640 1 -- 172.21.15.159:0/125743756 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f430c10c170 con 0x7f42f40a32d0 2024-09-19T07:33:01.084 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:00 smithi159 ceph-mon[30472]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:33:01.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:01.091+0000 7f430a000640 1 -- 172.21.15.159:0/125743756 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340801 (secure 0 0 0) 0x7f430c10c170 con 0x7f42f40a32d0 2024-09-19T07:33:01.095 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:33:01.097 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-09-19T07:33:01.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:01.096+0000 7f4312800640 1 -- 172.21.15.159:0/125743756 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f42f40a32d0 msgr2=0x7f42f40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:01.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:01.096+0000 7f4312800640 1 --2- 172.21.15.159:0/125743756 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f42f40a32d0 0x7f42f40a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4300009bd0 tx=0x7f4300005f90 comp rx=0 tx=0).stop 2024-09-19T07:33:01.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:01.096+0000 7f4312800640 1 -- 172.21.15.159:0/125743756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f430c1047e0 msgr2=0x7f430c19a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:01.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:01.096+0000 7f4312800640 1 --2- 172.21.15.159:0/125743756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f430c1047e0 0x7f430c19a9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f42f800e9f0 tx=0x7f42f800eeb0 comp rx=0 tx=0).stop 2024-09-19T07:33:01.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:01.097+0000 7f4312800640 1 -- 172.21.15.159:0/125743756 shutdown_connections 2024-09-19T07:33:01.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:01.097+0000 7f4312800640 1 --2- 172.21.15.159:0/125743756 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f42f40a32d0 0x7f42f40a5790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:01.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:01.097+0000 7f4312800640 1 --2- 172.21.15.159:0/125743756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f430c108f60 0x7f430c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:01.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:01.097+0000 7f4312800640 1 --2- 172.21.15.159:0/125743756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f430c1047e0 0x7f430c19a9c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:01.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:01.097+0000 7f4312800640 1 --2- 172.21.15.159:0/125743756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f430c103e30 0x7f430c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:01.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:01.097+0000 7f4312800640 1 -- 172.21.15.159:0/125743756 >> 172.21.15.159:0/125743756 conn(0x7f430c0ff910 msgr2=0x7f430c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:01.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:01.097+0000 7f4312800640 1 -- 172.21.15.159:0/125743756 shutdown_connections 2024-09-19T07:33:01.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:01.097+0000 7f4312800640 1 -- 172.21.15.159:0/125743756 wait complete. 2024-09-19T07:33:01.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:00 smithi203 ceph-mon[35335]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:33:01.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:00 smithi189 ceph-mon[34851]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:33:02.009 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-09-19T07:33:00.720563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656521,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":539,"num_read_kb":385,"num_write":348,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":354396,"kb_used_data":13388,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124587428,"statfs":{"total":1151940427776,"available":1151577526272,"internally_reserved":0,"allocated":13709312,"data_stored":8810189,"data_compressed":501084,"data_compressed_allocated":688128,"data_compressed_original":2875392,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003113"},"pg_stats":[{"pgid":"2.1f","version":"98'9","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186519+0000","last_change":"2024-09-19T07:30:59.408103+0000","last_active":"2024-09-19T07:32:32.186519+0000","last_peered":"2024-09-19T07:32:32.186519+0000","last_clean":"2024-09-19T07:32:32.186519+0000","last_became_active":"2024-09-19T07:30:59.407585+0000","last_became_peered":"2024-09-19T07:30:59.407585+0000","last_unstale":"2024-09-19T07:32:32.186519+0000","last_undegraded":"2024-09-19T07:32:32.186519+0000","last_fullsized":"2024-09-19T07:32:32.186519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:03.104939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841762+0000","last_change":"2024-09-19T07:31:02.438518+0000","last_active":"2024-09-19T07:32:31.841762+0000","last_peered":"2024-09-19T07:32:31.841762+0000","last_clean":"2024-09-19T07:32:31.841762+0000","last_became_active":"2024-09-19T07:31:02.438025+0000","last_became_peered":"2024-09-19T07:31:02.438025+0000","last_unstale":"2024-09-19T07:32:31.841762+0000","last_undegraded":"2024-09-19T07:32:31.841762+0000","last_fullsized":"2024-09-19T07:32:31.841762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:59.148172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839630+0000","last_change":"2024-09-19T07:31:12.503517+0000","last_active":"2024-09-19T07:32:31.839630+0000","last_peered":"2024-09-19T07:32:31.839630+0000","last_clean":"2024-09-19T07:32:31.839630+0000","last_became_active":"2024-09-19T07:31:12.503155+0000","last_became_peered":"2024-09-19T07:31:12.503155+0000","last_unstale":"2024-09-19T07:32:31.839630+0000","last_undegraded":"2024-09-19T07:32:31.839630+0000","last_fullsized":"2024-09-19T07:32:31.839630+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:15.475388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842008+0000","last_change":"2024-09-19T07:30:59.445107+0000","last_active":"2024-09-19T07:32:31.842008+0000","last_peered":"2024-09-19T07:32:31.842008+0000","last_clean":"2024-09-19T07:32:31.842008+0000","last_became_active":"2024-09-19T07:30:59.444782+0000","last_became_peered":"2024-09-19T07:30:59.444782+0000","last_unstale":"2024-09-19T07:32:31.842008+0000","last_undegraded":"2024-09-19T07:32:31.842008+0000","last_fullsized":"2024-09-19T07:32:31.842008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:37.582314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186057+0000","last_change":"2024-09-19T07:31:02.437690+0000","last_active":"2024-09-19T07:32:32.186057+0000","last_peered":"2024-09-19T07:32:32.186057+0000","last_clean":"2024-09-19T07:32:32.186057+0000","last_became_active":"2024-09-19T07:31:02.429969+0000","last_became_peered":"2024-09-19T07:31:02.429969+0000","last_unstale":"2024-09-19T07:32:32.186057+0000","last_undegraded":"2024-09-19T07:32:32.186057+0000","last_fullsized":"2024-09-19T07:32:32.186057+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:23.227114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843148+0000","last_change":"2024-09-19T07:31:12.500314+0000","last_active":"2024-09-19T07:32:31.843148+0000","last_peered":"2024-09-19T07:32:31.843148+0000","last_clean":"2024-09-19T07:32:31.843148+0000","last_became_active":"2024-09-19T07:31:12.499529+0000","last_became_peered":"2024-09-19T07:31:12.499529+0000","last_unstale":"2024-09-19T07:32:31.843148+0000","last_undegraded":"2024-09-19T07:32:31.843148+0000","last_fullsized":"2024-09-19T07:32:31.843148+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:46.805336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.844209+0000","last_change":"2024-09-19T07:30:59.408974+0000","last_active":"2024-09-19T07:32:31.844209+0000","last_peered":"2024-09-19T07:32:31.844209+0000","last_clean":"2024-09-19T07:32:31.844209+0000","last_became_active":"2024-09-19T07:30:59.408606+0000","last_became_peered":"2024-09-19T07:30:59.408606+0000","last_unstale":"2024-09-19T07:32:31.844209+0000","last_undegraded":"2024-09-19T07:32:31.844209+0000","last_fullsized":"2024-09-19T07:32:31.844209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:21.962123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185445+0000","last_change":"2024-09-19T07:31:02.431202+0000","last_active":"2024-09-19T07:32:32.185445+0000","last_peered":"2024-09-19T07:32:32.185445+0000","last_clean":"2024-09-19T07:32:32.185445+0000","last_became_active":"2024-09-19T07:31:02.430789+0000","last_became_peered":"2024-09-19T07:31:02.430789+0000","last_unstale":"2024-09-19T07:32:32.185445+0000","last_undegraded":"2024-09-19T07:32:32.185445+0000","last_fullsized":"2024-09-19T07:32:32.185445+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:40.750108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843603+0000","last_change":"2024-09-19T07:31:12.500000+0000","last_active":"2024-09-19T07:32:31.843603+0000","last_peered":"2024-09-19T07:32:31.843603+0000","last_clean":"2024-09-19T07:32:31.843603+0000","last_became_active":"2024-09-19T07:31:12.499526+0000","last_became_peered":"2024-09-19T07:31:12.499526+0000","last_unstale":"2024-09-19T07:32:31.843603+0000","last_undegraded":"2024-09-19T07:32:31.843603+0000","last_fullsized":"2024-09-19T07:32:31.843603+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:15.449391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843823+0000","last_change":"2024-09-19T07:30:59.411645+0000","last_active":"2024-09-19T07:32:31.843823+0000","last_peered":"2024-09-19T07:32:31.843823+0000","last_clean":"2024-09-19T07:32:31.843823+0000","last_became_active":"2024-09-19T07:30:59.411262+0000","last_became_peered":"2024-09-19T07:30:59.411262+0000","last_unstale":"2024-09-19T07:32:31.843823+0000","last_undegraded":"2024-09-19T07:32:31.843823+0000","last_fullsized":"2024-09-19T07:32:31.843823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:53.538203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842984+0000","last_change":"2024-09-19T07:31:32.247958+0000","last_active":"2024-09-19T07:32:31.842984+0000","last_peered":"2024-09-19T07:32:31.842984+0000","last_clean":"2024-09-19T07:32:31.842984+0000","last_became_active":"2024-09-19T07:31:32.247521+0000","last_became_peered":"2024-09-19T07:31:32.247521+0000","last_unstale":"2024-09-19T07:32:31.842984+0000","last_undegraded":"2024-09-19T07:32:31.842984+0000","last_fullsized":"2024-09-19T07:32:31.842984+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:38.291915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842725+0000","last_change":"2024-09-19T07:31:12.500070+0000","last_active":"2024-09-19T07:32:31.842725+0000","last_peered":"2024-09-19T07:32:31.842725+0000","last_clean":"2024-09-19T07:32:31.842725+0000","last_became_active":"2024-09-19T07:31:12.499658+0000","last_became_peered":"2024-09-19T07:31:12.499658+0000","last_unstale":"2024-09-19T07:32:31.842725+0000","last_undegraded":"2024-09-19T07:32:31.842725+0000","last_fullsized":"2024-09-19T07:32:31.842725+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:08.955749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840047+0000","last_change":"2024-09-19T07:31:02.426972+0000","last_active":"2024-09-19T07:32:31.840047+0000","last_peered":"2024-09-19T07:32:31.840047+0000","last_clean":"2024-09-19T07:32:31.840047+0000","last_became_active":"2024-09-19T07:31:02.426661+0000","last_became_peered":"2024-09-19T07:31:02.426661+0000","last_unstale":"2024-09-19T07:32:31.840047+0000","last_undegraded":"2024-09-19T07:32:31.840047+0000","last_fullsized":"2024-09-19T07:32:31.840047+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:26.943324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842989+0000","last_change":"2024-09-19T07:30:59.410056+0000","last_active":"2024-09-19T07:32:31.842989+0000","last_peered":"2024-09-19T07:32:31.842989+0000","last_clean":"2024-09-19T07:32:31.842989+0000","last_became_active":"2024-09-19T07:30:59.409669+0000","last_became_peered":"2024-09-19T07:30:59.409669+0000","last_unstale":"2024-09-19T07:32:31.842989+0000","last_undegraded":"2024-09-19T07:32:31.842989+0000","last_fullsized":"2024-09-19T07:32:31.842989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:43.169905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842918+0000","last_change":"2024-09-19T07:31:12.557969+0000","last_active":"2024-09-19T07:32:31.842918+0000","last_peered":"2024-09-19T07:32:31.842918+0000","last_clean":"2024-09-19T07:32:31.842918+0000","last_became_active":"2024-09-19T07:31:12.556692+0000","last_became_peered":"2024-09-19T07:31:12.556692+0000","last_unstale":"2024-09-19T07:32:31.842918+0000","last_undegraded":"2024-09-19T07:32:31.842918+0000","last_fullsized":"2024-09-19T07:32:31.842918+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.563341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841908+0000","last_change":"2024-09-19T07:31:02.438979+0000","last_active":"2024-09-19T07:32:31.841908+0000","last_peered":"2024-09-19T07:32:31.841908+0000","last_clean":"2024-09-19T07:32:31.841908+0000","last_became_active":"2024-09-19T07:31:02.437192+0000","last_became_peered":"2024-09-19T07:31:02.437192+0000","last_unstale":"2024-09-19T07:32:31.841908+0000","last_undegraded":"2024-09-19T07:32:31.841908+0000","last_fullsized":"2024-09-19T07:32:31.841908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:39.957366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839831+0000","last_change":"2024-09-19T07:30:59.406970+0000","last_active":"2024-09-19T07:32:31.839831+0000","last_peered":"2024-09-19T07:32:31.839831+0000","last_clean":"2024-09-19T07:32:31.839831+0000","last_became_active":"2024-09-19T07:30:59.406537+0000","last_became_peered":"2024-09-19T07:30:59.406537+0000","last_unstale":"2024-09-19T07:32:31.839831+0000","last_undegraded":"2024-09-19T07:32:31.839831+0000","last_fullsized":"2024-09-19T07:32:31.839831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:32.969944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842829+0000","last_change":"2024-09-19T07:31:12.495709+0000","last_active":"2024-09-19T07:32:31.842829+0000","last_peered":"2024-09-19T07:32:31.842829+0000","last_clean":"2024-09-19T07:32:31.842829+0000","last_became_active":"2024-09-19T07:31:12.495201+0000","last_became_peered":"2024-09-19T07:31:12.495201+0000","last_unstale":"2024-09-19T07:32:31.842829+0000","last_undegraded":"2024-09-19T07:32:31.842829+0000","last_fullsized":"2024-09-19T07:32:31.842829+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:48.710011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839225+0000","last_change":"2024-09-19T07:31:02.431032+0000","last_active":"2024-09-19T07:32:31.839225+0000","last_peered":"2024-09-19T07:32:31.839225+0000","last_clean":"2024-09-19T07:32:31.839225+0000","last_became_active":"2024-09-19T07:31:02.430691+0000","last_became_peered":"2024-09-19T07:31:02.430691+0000","last_unstale":"2024-09-19T07:32:31.839225+0000","last_undegraded":"2024-09-19T07:32:31.839225+0000","last_fullsized":"2024-09-19T07:32:31.839225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:59.627125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841894+0000","last_change":"2024-09-19T07:30:59.410931+0000","last_active":"2024-09-19T07:32:31.841894+0000","last_peered":"2024-09-19T07:32:31.841894+0000","last_clean":"2024-09-19T07:32:31.841894+0000","last_became_active":"2024-09-19T07:30:59.410516+0000","last_became_peered":"2024-09-19T07:30:59.410516+0000","last_unstale":"2024-09-19T07:32:31.841894+0000","last_undegraded":"2024-09-19T07:32:31.841894+0000","last_fullsized":"2024-09-19T07:32:31.841894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:39.688441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.837387+0000","last_change":"2024-09-19T07:31:31.809224+0000","last_active":"2024-09-19T07:32:31.837387+0000","last_peered":"2024-09-19T07:32:31.837387+0000","last_clean":"2024-09-19T07:32:31.837387+0000","last_became_active":"2024-09-19T07:31:31.808673+0000","last_became_peered":"2024-09-19T07:31:31.808673+0000","last_unstale":"2024-09-19T07:32:31.837387+0000","last_undegraded":"2024-09-19T07:32:31.837387+0000","last_fullsized":"2024-09-19T07:32:31.837387+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:38.675403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842511+0000","last_change":"2024-09-19T07:31:02.426735+0000","last_active":"2024-09-19T07:32:31.842511+0000","last_peered":"2024-09-19T07:32:31.842511+0000","last_clean":"2024-09-19T07:32:31.842511+0000","last_became_active":"2024-09-19T07:31:02.426324+0000","last_became_peered":"2024-09-19T07:31:02.426324+0000","last_unstale":"2024-09-19T07:32:31.842511+0000","last_undegraded":"2024-09-19T07:32:31.842511+0000","last_fullsized":"2024-09-19T07:32:31.842511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:10.171191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185274+0000","last_change":"2024-09-19T07:30:59.445125+0000","last_active":"2024-09-19T07:32:32.185274+0000","last_peered":"2024-09-19T07:32:32.185274+0000","last_clean":"2024-09-19T07:32:32.185274+0000","last_became_active":"2024-09-19T07:30:59.444757+0000","last_became_peered":"2024-09-19T07:30:59.444757+0000","last_unstale":"2024-09-19T07:32:32.185274+0000","last_undegraded":"2024-09-19T07:32:32.185274+0000","last_fullsized":"2024-09-19T07:32:32.185274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:34.183389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843133+0000","last_change":"2024-09-19T07:31:12.557277+0000","last_active":"2024-09-19T07:32:31.843133+0000","last_peered":"2024-09-19T07:32:31.843133+0000","last_clean":"2024-09-19T07:32:31.843133+0000","last_became_active":"2024-09-19T07:31:12.556885+0000","last_became_peered":"2024-09-19T07:31:12.556885+0000","last_unstale":"2024-09-19T07:32:31.843133+0000","last_undegraded":"2024-09-19T07:32:31.843133+0000","last_fullsized":"2024-09-19T07:32:31.843133+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:39.960404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186040+0000","last_change":"2024-09-19T07:31:02.441712+0000","last_active":"2024-09-19T07:32:32.186040+0000","last_peered":"2024-09-19T07:32:32.186040+0000","last_clean":"2024-09-19T07:32:32.186040+0000","last_became_active":"2024-09-19T07:31:02.440039+0000","last_became_peered":"2024-09-19T07:31:02.440039+0000","last_unstale":"2024-09-19T07:32:32.186040+0000","last_undegraded":"2024-09-19T07:32:32.186040+0000","last_fullsized":"2024-09-19T07:32:32.186040+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:13.601774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843256+0000","last_change":"2024-09-19T07:31:00.027265+0000","last_active":"2024-09-19T07:32:31.843256+0000","last_peered":"2024-09-19T07:32:31.843256+0000","last_clean":"2024-09-19T07:32:31.843256+0000","last_became_active":"2024-09-19T07:31:00.026924+0000","last_became_peered":"2024-09-19T07:31:00.026924+0000","last_unstale":"2024-09-19T07:32:31.843256+0000","last_undegraded":"2024-09-19T07:32:31.843256+0000","last_fullsized":"2024-09-19T07:32:31.843256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:05.290978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839599+0000","last_change":"2024-09-19T07:31:12.495613+0000","last_active":"2024-09-19T07:32:31.839599+0000","last_peered":"2024-09-19T07:32:31.839599+0000","last_clean":"2024-09-19T07:32:31.839599+0000","last_became_active":"2024-09-19T07:31:12.495218+0000","last_became_peered":"2024-09-19T07:31:12.495218+0000","last_unstale":"2024-09-19T07:32:31.839599+0000","last_undegraded":"2024-09-19T07:32:31.839599+0000","last_fullsized":"2024-09-19T07:32:31.839599+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:27.651671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841839+0000","last_change":"2024-09-19T07:31:02.442902+0000","last_active":"2024-09-19T07:32:31.841839+0000","last_peered":"2024-09-19T07:32:31.841839+0000","last_clean":"2024-09-19T07:32:31.841839+0000","last_became_active":"2024-09-19T07:31:02.437173+0000","last_became_peered":"2024-09-19T07:31:02.437173+0000","last_unstale":"2024-09-19T07:32:31.841839+0000","last_undegraded":"2024-09-19T07:32:31.841839+0000","last_fullsized":"2024-09-19T07:32:31.841839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:40:44.760641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'10","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186477+0000","last_change":"2024-09-19T07:30:59.406488+0000","last_active":"2024-09-19T07:32:32.186477+0000","last_peered":"2024-09-19T07:32:32.186477+0000","last_clean":"2024-09-19T07:32:32.186477+0000","last_became_active":"2024-09-19T07:30:59.406121+0000","last_became_peered":"2024-09-19T07:30:59.406121+0000","last_unstale":"2024-09-19T07:32:32.186477+0000","last_undegraded":"2024-09-19T07:32:32.186477+0000","last_fullsized":"2024-09-19T07:32:32.186477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:14:50.534009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841342+0000","last_change":"2024-09-19T07:31:12.558127+0000","last_active":"2024-09-19T07:32:31.841342+0000","last_peered":"2024-09-19T07:32:31.841342+0000","last_clean":"2024-09-19T07:32:31.841342+0000","last_became_active":"2024-09-19T07:31:12.557078+0000","last_became_peered":"2024-09-19T07:31:12.557078+0000","last_unstale":"2024-09-19T07:32:31.841342+0000","last_undegraded":"2024-09-19T07:32:31.841342+0000","last_fullsized":"2024-09-19T07:32:31.841342+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:23.076116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836615+0000","last_change":"2024-09-19T07:31:02.433918+0000","last_active":"2024-09-19T07:32:31.836615+0000","last_peered":"2024-09-19T07:32:31.836615+0000","last_clean":"2024-09-19T07:32:31.836615+0000","last_became_active":"2024-09-19T07:31:02.430802+0000","last_became_peered":"2024-09-19T07:31:02.430802+0000","last_unstale":"2024-09-19T07:32:31.836615+0000","last_undegraded":"2024-09-19T07:32:31.836615+0000","last_fullsized":"2024-09-19T07:32:31.836615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:28.948441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.844215+0000","last_change":"2024-09-19T07:30:59.444178+0000","last_active":"2024-09-19T07:32:31.844215+0000","last_peered":"2024-09-19T07:32:31.844215+0000","last_clean":"2024-09-19T07:32:31.844215+0000","last_became_active":"2024-09-19T07:30:59.443847+0000","last_became_peered":"2024-09-19T07:30:59.443847+0000","last_unstale":"2024-09-19T07:32:31.844215+0000","last_undegraded":"2024-09-19T07:32:31.844215+0000","last_fullsized":"2024-09-19T07:32:31.844215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:08.588244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839855+0000","last_change":"2024-09-19T07:31:12.508847+0000","last_active":"2024-09-19T07:32:31.839855+0000","last_peered":"2024-09-19T07:32:31.839855+0000","last_clean":"2024-09-19T07:32:31.839855+0000","last_became_active":"2024-09-19T07:31:12.508371+0000","last_became_peered":"2024-09-19T07:31:12.508371+0000","last_unstale":"2024-09-19T07:32:31.839855+0000","last_undegraded":"2024-09-19T07:32:31.839855+0000","last_fullsized":"2024-09-19T07:32:31.839855+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:27.420724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.837169+0000","last_change":"2024-09-19T07:31:02.438213+0000","last_active":"2024-09-19T07:32:31.837169+0000","last_peered":"2024-09-19T07:32:31.837169+0000","last_clean":"2024-09-19T07:32:31.837169+0000","last_became_active":"2024-09-19T07:31:02.437892+0000","last_became_peered":"2024-09-19T07:31:02.437892+0000","last_unstale":"2024-09-19T07:32:31.837169+0000","last_undegraded":"2024-09-19T07:32:31.837169+0000","last_fullsized":"2024-09-19T07:32:31.837169+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:50.249962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186288+0000","last_change":"2024-09-19T07:30:59.445523+0000","last_active":"2024-09-19T07:32:32.186288+0000","last_peered":"2024-09-19T07:32:32.186288+0000","last_clean":"2024-09-19T07:32:32.186288+0000","last_became_active":"2024-09-19T07:30:59.445118+0000","last_became_peered":"2024-09-19T07:30:59.445118+0000","last_unstale":"2024-09-19T07:32:32.186288+0000","last_undegraded":"2024-09-19T07:32:32.186288+0000","last_fullsized":"2024-09-19T07:32:32.186288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:35.369102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842543+0000","last_change":"2024-09-19T07:31:12.557541+0000","last_active":"2024-09-19T07:32:31.842543+0000","last_peered":"2024-09-19T07:32:31.842543+0000","last_clean":"2024-09-19T07:32:31.842543+0000","last_became_active":"2024-09-19T07:31:12.557130+0000","last_became_peered":"2024-09-19T07:31:12.557130+0000","last_unstale":"2024-09-19T07:32:31.842543+0000","last_undegraded":"2024-09-19T07:32:31.842543+0000","last_fullsized":"2024-09-19T07:32:31.842543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:52.610222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.837091+0000","last_change":"2024-09-19T07:31:02.434743+0000","last_active":"2024-09-19T07:32:31.837091+0000","last_peered":"2024-09-19T07:32:31.837091+0000","last_clean":"2024-09-19T07:32:31.837091+0000","last_became_active":"2024-09-19T07:31:02.434312+0000","last_became_peered":"2024-09-19T07:31:02.434312+0000","last_unstale":"2024-09-19T07:32:31.837091+0000","last_undegraded":"2024-09-19T07:32:31.837091+0000","last_fullsized":"2024-09-19T07:32:31.837091+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:24:00.569195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840169+0000","last_change":"2024-09-19T07:30:59.409893+0000","last_active":"2024-09-19T07:32:31.840169+0000","last_peered":"2024-09-19T07:32:31.840169+0000","last_clean":"2024-09-19T07:32:31.840169+0000","last_became_active":"2024-09-19T07:30:59.409440+0000","last_became_peered":"2024-09-19T07:30:59.409440+0000","last_unstale":"2024-09-19T07:32:31.840169+0000","last_undegraded":"2024-09-19T07:32:31.840169+0000","last_fullsized":"2024-09-19T07:32:31.840169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:45.999596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836583+0000","last_change":"2024-09-19T07:31:12.505965+0000","last_active":"2024-09-19T07:32:31.836583+0000","last_peered":"2024-09-19T07:32:31.836583+0000","last_clean":"2024-09-19T07:32:31.836583+0000","last_became_active":"2024-09-19T07:31:12.504675+0000","last_became_peered":"2024-09-19T07:31:12.504675+0000","last_unstale":"2024-09-19T07:32:31.836583+0000","last_undegraded":"2024-09-19T07:32:31.836583+0000","last_fullsized":"2024-09-19T07:32:31.836583+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:47.345020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842290+0000","last_change":"2024-09-19T07:31:31.809488+0000","last_active":"2024-09-19T07:32:31.842290+0000","last_peered":"2024-09-19T07:32:31.842290+0000","last_clean":"2024-09-19T07:32:31.842290+0000","last_became_active":"2024-09-19T07:31:31.809107+0000","last_became_peered":"2024-09-19T07:31:31.809107+0000","last_unstale":"2024-09-19T07:32:31.842290+0000","last_undegraded":"2024-09-19T07:32:31.842290+0000","last_fullsized":"2024-09-19T07:32:31.842290+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:39.757841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":114,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:55.993060+0000","last_change":"2024-09-19T07:30:59.445001+0000","last_active":"2024-09-19T07:32:55.993060+0000","last_peered":"2024-09-19T07:32:55.993060+0000","last_clean":"2024-09-19T07:32:55.993060+0000","last_became_active":"2024-09-19T07:30:59.444590+0000","last_became_peered":"2024-09-19T07:30:59.444590+0000","last_unstale":"2024-09-19T07:32:55.993060+0000","last_undegraded":"2024-09-19T07:32:55.993060+0000","last_fullsized":"2024-09-19T07:32:55.993060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:13.934085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836988+0000","last_change":"2024-09-19T07:31:12.557622+0000","last_active":"2024-09-19T07:32:31.836988+0000","last_peered":"2024-09-19T07:32:31.836988+0000","last_clean":"2024-09-19T07:32:31.836988+0000","last_became_active":"2024-09-19T07:31:12.556934+0000","last_became_peered":"2024-09-19T07:31:12.556934+0000","last_unstale":"2024-09-19T07:32:31.836988+0000","last_undegraded":"2024-09-19T07:32:31.836988+0000","last_fullsized":"2024-09-19T07:32:31.836988+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:06:31.944819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836423+0000","last_change":"2024-09-19T07:31:02.433199+0000","last_active":"2024-09-19T07:32:31.836423+0000","last_peered":"2024-09-19T07:32:31.836423+0000","last_clean":"2024-09-19T07:32:31.836423+0000","last_became_active":"2024-09-19T07:31:02.432739+0000","last_became_peered":"2024-09-19T07:31:02.432739+0000","last_unstale":"2024-09-19T07:32:31.836423+0000","last_undegraded":"2024-09-19T07:32:31.836423+0000","last_fullsized":"2024-09-19T07:32:31.836423+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:57.392885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.849682+0000","last_change":"2024-09-19T07:32:31.849682+0000","last_active":"2024-09-19T07:32:31.849682+0000","last_peered":"2024-09-19T07:32:31.849682+0000","last_clean":"2024-09-19T07:32:31.849682+0000","last_became_active":"2024-09-19T07:32:31.846644+0000","last_became_peered":"2024-09-19T07:32:31.846644+0000","last_unstale":"2024-09-19T07:32:31.849682+0000","last_undegraded":"2024-09-19T07:32:31.849682+0000","last_fullsized":"2024-09-19T07:32:31.849682+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:14.117168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843173+0000","last_change":"2024-09-19T07:31:12.557638+0000","last_active":"2024-09-19T07:32:31.843173+0000","last_peered":"2024-09-19T07:32:31.843173+0000","last_clean":"2024-09-19T07:32:31.843173+0000","last_became_active":"2024-09-19T07:31:12.557170+0000","last_became_peered":"2024-09-19T07:31:12.557170+0000","last_unstale":"2024-09-19T07:32:31.843173+0000","last_undegraded":"2024-09-19T07:32:31.843173+0000","last_fullsized":"2024-09-19T07:32:31.843173+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:14.077038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841823+0000","last_change":"2024-09-19T07:31:02.443081+0000","last_active":"2024-09-19T07:32:31.841823+0000","last_peered":"2024-09-19T07:32:31.841823+0000","last_clean":"2024-09-19T07:32:31.841823+0000","last_became_active":"2024-09-19T07:31:02.437034+0000","last_became_peered":"2024-09-19T07:31:02.437034+0000","last_unstale":"2024-09-19T07:32:31.841823+0000","last_undegraded":"2024-09-19T07:32:31.841823+0000","last_fullsized":"2024-09-19T07:32:31.841823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:55.310844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843769+0000","last_change":"2024-09-19T07:30:59.410069+0000","last_active":"2024-09-19T07:32:31.843769+0000","last_peered":"2024-09-19T07:32:31.843769+0000","last_clean":"2024-09-19T07:32:31.843769+0000","last_became_active":"2024-09-19T07:30:59.409736+0000","last_became_peered":"2024-09-19T07:30:59.409736+0000","last_unstale":"2024-09-19T07:32:31.843769+0000","last_undegraded":"2024-09-19T07:32:31.843769+0000","last_fullsized":"2024-09-19T07:32:31.843769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:59:47.095227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":361,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840287+0000","last_change":"2024-09-19T07:29:38.994670+0000","last_active":"2024-09-19T07:32:31.840287+0000","last_peered":"2024-09-19T07:32:31.840287+0000","last_clean":"2024-09-19T07:32:31.840287+0000","last_became_active":"2024-09-19T07:29:38.994229+0000","last_became_peered":"2024-09-19T07:29:38.994229+0000","last_unstale":"2024-09-19T07:32:31.840287+0000","last_undegraded":"2024-09-19T07:32:31.840287+0000","last_fullsized":"2024-09-19T07:32:31.840287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:25:55.396577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:25:55.396577+0000","last_clean_scrub_stamp":"2024-09-19T07:25:55.396577+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:06.972254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840232+0000","last_change":"2024-09-19T07:31:12.497060+0000","last_active":"2024-09-19T07:32:31.840232+0000","last_peered":"2024-09-19T07:32:31.840232+0000","last_clean":"2024-09-19T07:32:31.840232+0000","last_became_active":"2024-09-19T07:31:12.496693+0000","last_became_peered":"2024-09-19T07:31:12.496693+0000","last_unstale":"2024-09-19T07:32:31.840232+0000","last_undegraded":"2024-09-19T07:32:31.840232+0000","last_fullsized":"2024-09-19T07:32:31.840232+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:41.879585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840496+0000","last_change":"2024-09-19T07:31:02.420732+0000","last_active":"2024-09-19T07:32:31.840496+0000","last_peered":"2024-09-19T07:32:31.840496+0000","last_clean":"2024-09-19T07:32:31.840496+0000","last_became_active":"2024-09-19T07:31:02.420394+0000","last_became_peered":"2024-09-19T07:31:02.420394+0000","last_unstale":"2024-09-19T07:32:31.840496+0000","last_undegraded":"2024-09-19T07:32:31.840496+0000","last_fullsized":"2024-09-19T07:32:31.840496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:48:36.157189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839893+0000","last_change":"2024-09-19T07:31:00.027344+0000","last_active":"2024-09-19T07:32:31.839893+0000","last_peered":"2024-09-19T07:32:31.839893+0000","last_clean":"2024-09-19T07:32:31.839893+0000","last_became_active":"2024-09-19T07:31:00.026855+0000","last_became_peered":"2024-09-19T07:31:00.026855+0000","last_unstale":"2024-09-19T07:32:31.839893+0000","last_undegraded":"2024-09-19T07:32:31.839893+0000","last_fullsized":"2024-09-19T07:32:31.839893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:12.304161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185188+0000","last_change":"2024-09-19T07:31:12.495883+0000","last_active":"2024-09-19T07:32:32.185188+0000","last_peered":"2024-09-19T07:32:32.185188+0000","last_clean":"2024-09-19T07:32:32.185188+0000","last_became_active":"2024-09-19T07:31:12.495253+0000","last_became_peered":"2024-09-19T07:31:12.495253+0000","last_unstale":"2024-09-19T07:32:32.185188+0000","last_undegraded":"2024-09-19T07:32:32.185188+0000","last_fullsized":"2024-09-19T07:32:32.185188+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:56.608977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840208+0000","last_change":"2024-09-19T07:31:02.427706+0000","last_active":"2024-09-19T07:32:31.840208+0000","last_peered":"2024-09-19T07:32:31.840208+0000","last_clean":"2024-09-19T07:32:31.840208+0000","last_became_active":"2024-09-19T07:31:02.427398+0000","last_became_peered":"2024-09-19T07:31:02.427398+0000","last_unstale":"2024-09-19T07:32:31.840208+0000","last_undegraded":"2024-09-19T07:32:31.840208+0000","last_fullsized":"2024-09-19T07:32:31.840208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:13.179472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843207+0000","last_change":"2024-09-19T07:30:59.410020+0000","last_active":"2024-09-19T07:32:31.843207+0000","last_peered":"2024-09-19T07:32:31.843207+0000","last_clean":"2024-09-19T07:32:31.843207+0000","last_became_active":"2024-09-19T07:30:59.409633+0000","last_became_peered":"2024-09-19T07:30:59.409633+0000","last_unstale":"2024-09-19T07:32:31.843207+0000","last_undegraded":"2024-09-19T07:32:31.843207+0000","last_fullsized":"2024-09-19T07:32:31.843207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:49.673952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185781+0000","last_change":"2024-09-19T07:31:12.511259+0000","last_active":"2024-09-19T07:32:32.185781+0000","last_peered":"2024-09-19T07:32:32.185781+0000","last_clean":"2024-09-19T07:32:32.185781+0000","last_became_active":"2024-09-19T07:31:12.510691+0000","last_became_peered":"2024-09-19T07:31:12.510691+0000","last_unstale":"2024-09-19T07:32:32.185781+0000","last_undegraded":"2024-09-19T07:32:32.185781+0000","last_fullsized":"2024-09-19T07:32:32.185781+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:09.388922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840526+0000","last_change":"2024-09-19T07:31:02.425922+0000","last_active":"2024-09-19T07:32:31.840526+0000","last_peered":"2024-09-19T07:32:31.840526+0000","last_clean":"2024-09-19T07:32:31.840526+0000","last_became_active":"2024-09-19T07:31:02.425565+0000","last_became_peered":"2024-09-19T07:31:02.425565+0000","last_unstale":"2024-09-19T07:32:31.840526+0000","last_undegraded":"2024-09-19T07:32:31.840526+0000","last_fullsized":"2024-09-19T07:32:31.840526+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:48.997362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836916+0000","last_change":"2024-09-19T07:31:31.813529+0000","last_active":"2024-09-19T07:32:31.836916+0000","last_peered":"2024-09-19T07:32:31.836916+0000","last_clean":"2024-09-19T07:32:31.836916+0000","last_became_active":"2024-09-19T07:31:31.809205+0000","last_became_peered":"2024-09-19T07:31:31.809205+0000","last_unstale":"2024-09-19T07:32:31.836916+0000","last_undegraded":"2024-09-19T07:32:31.836916+0000","last_fullsized":"2024-09-19T07:32:31.836916+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:26.117905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841916+0000","last_change":"2024-09-19T07:31:12.497728+0000","last_active":"2024-09-19T07:32:31.841916+0000","last_peered":"2024-09-19T07:32:31.841916+0000","last_clean":"2024-09-19T07:32:31.841916+0000","last_became_active":"2024-09-19T07:31:12.497315+0000","last_became_peered":"2024-09-19T07:31:12.497315+0000","last_unstale":"2024-09-19T07:32:31.841916+0000","last_undegraded":"2024-09-19T07:32:31.841916+0000","last_fullsized":"2024-09-19T07:32:31.841916+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:12.174147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839628+0000","last_change":"2024-09-19T07:31:02.433205+0000","last_active":"2024-09-19T07:32:31.839628+0000","last_peered":"2024-09-19T07:32:31.839628+0000","last_clean":"2024-09-19T07:32:31.839628+0000","last_became_active":"2024-09-19T07:31:02.432856+0000","last_became_peered":"2024-09-19T07:31:02.432856+0000","last_unstale":"2024-09-19T07:32:31.839628+0000","last_undegraded":"2024-09-19T07:32:31.839628+0000","last_fullsized":"2024-09-19T07:32:31.839628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:31.470420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.856388+0000","last_change":"2024-09-19T07:32:31.855914+0000","last_active":"2024-09-19T07:32:31.856388+0000","last_peered":"2024-09-19T07:32:31.856388+0000","last_clean":"2024-09-19T07:32:31.856388+0000","last_became_active":"2024-09-19T07:32:31.844841+0000","last_became_peered":"2024-09-19T07:32:31.844841+0000","last_unstale":"2024-09-19T07:32:31.856388+0000","last_undegraded":"2024-09-19T07:32:31.856388+0000","last_fullsized":"2024-09-19T07:32:31.856388+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:57.051556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841966+0000","last_change":"2024-09-19T07:31:12.500715+0000","last_active":"2024-09-19T07:32:31.841966+0000","last_peered":"2024-09-19T07:32:31.841966+0000","last_clean":"2024-09-19T07:32:31.841966+0000","last_became_active":"2024-09-19T07:31:12.500401+0000","last_became_peered":"2024-09-19T07:31:12.500401+0000","last_unstale":"2024-09-19T07:32:31.841966+0000","last_undegraded":"2024-09-19T07:32:31.841966+0000","last_fullsized":"2024-09-19T07:32:31.841966+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:59.158518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842282+0000","last_change":"2024-09-19T07:31:02.429062+0000","last_active":"2024-09-19T07:32:31.842282+0000","last_peered":"2024-09-19T07:32:31.842282+0000","last_clean":"2024-09-19T07:32:31.842282+0000","last_became_active":"2024-09-19T07:31:02.428703+0000","last_became_peered":"2024-09-19T07:31:02.428703+0000","last_unstale":"2024-09-19T07:32:31.842282+0000","last_undegraded":"2024-09-19T07:32:31.842282+0000","last_fullsized":"2024-09-19T07:32:31.842282+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:30.928698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842764+0000","last_change":"2024-09-19T07:30:59.444883+0000","last_active":"2024-09-19T07:32:31.842764+0000","last_peered":"2024-09-19T07:32:31.842764+0000","last_clean":"2024-09-19T07:32:31.842764+0000","last_became_active":"2024-09-19T07:30:59.444445+0000","last_became_peered":"2024-09-19T07:30:59.444445+0000","last_unstale":"2024-09-19T07:32:31.842764+0000","last_undegraded":"2024-09-19T07:32:31.842764+0000","last_fullsized":"2024-09-19T07:32:31.842764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:29:11.166450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836866+0000","last_change":"2024-09-19T07:31:12.508431+0000","last_active":"2024-09-19T07:32:31.836866+0000","last_peered":"2024-09-19T07:32:31.836866+0000","last_clean":"2024-09-19T07:32:31.836866+0000","last_became_active":"2024-09-19T07:31:12.507826+0000","last_became_peered":"2024-09-19T07:31:12.507826+0000","last_unstale":"2024-09-19T07:32:31.836866+0000","last_undegraded":"2024-09-19T07:32:31.836866+0000","last_fullsized":"2024-09-19T07:32:31.836866+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:59.545418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839874+0000","last_change":"2024-09-19T07:31:02.423973+0000","last_active":"2024-09-19T07:32:31.839874+0000","last_peered":"2024-09-19T07:32:31.839874+0000","last_clean":"2024-09-19T07:32:31.839874+0000","last_became_active":"2024-09-19T07:31:02.423609+0000","last_became_peered":"2024-09-19T07:31:02.423609+0000","last_unstale":"2024-09-19T07:32:31.839874+0000","last_undegraded":"2024-09-19T07:32:31.839874+0000","last_fullsized":"2024-09-19T07:32:31.839874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:23:47.693286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"98'4","reported_seq":115,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:54.992259+0000","last_change":"2024-09-19T07:30:59.412862+0000","last_active":"2024-09-19T07:32:54.992259+0000","last_peered":"2024-09-19T07:32:54.992259+0000","last_clean":"2024-09-19T07:32:54.992259+0000","last_became_active":"2024-09-19T07:30:59.412401+0000","last_became_peered":"2024-09-19T07:30:59.412401+0000","last_unstale":"2024-09-19T07:32:54.992259+0000","last_undegraded":"2024-09-19T07:32:54.992259+0000","last_fullsized":"2024-09-19T07:32:54.992259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:49.461312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186408+0000","last_change":"2024-09-19T07:31:12.507714+0000","last_active":"2024-09-19T07:32:32.186408+0000","last_peered":"2024-09-19T07:32:32.186408+0000","last_clean":"2024-09-19T07:32:32.186408+0000","last_became_active":"2024-09-19T07:31:12.506054+0000","last_became_peered":"2024-09-19T07:31:12.506054+0000","last_unstale":"2024-09-19T07:32:32.186408+0000","last_undegraded":"2024-09-19T07:32:32.186408+0000","last_fullsized":"2024-09-19T07:32:32.186408+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:18.070858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840107+0000","last_change":"2024-09-19T07:31:02.424718+0000","last_active":"2024-09-19T07:32:31.840107+0000","last_peered":"2024-09-19T07:32:31.840107+0000","last_clean":"2024-09-19T07:32:31.840107+0000","last_became_active":"2024-09-19T07:31:02.424351+0000","last_became_peered":"2024-09-19T07:31:02.424351+0000","last_unstale":"2024-09-19T07:32:31.840107+0000","last_undegraded":"2024-09-19T07:32:31.840107+0000","last_fullsized":"2024-09-19T07:32:31.840107+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:59.396752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842555+0000","last_change":"2024-09-19T07:30:59.410551+0000","last_active":"2024-09-19T07:32:31.842555+0000","last_peered":"2024-09-19T07:32:31.842555+0000","last_clean":"2024-09-19T07:32:31.842555+0000","last_became_active":"2024-09-19T07:30:59.410226+0000","last_became_peered":"2024-09-19T07:30:59.410226+0000","last_unstale":"2024-09-19T07:32:31.842555+0000","last_undegraded":"2024-09-19T07:32:31.842555+0000","last_fullsized":"2024-09-19T07:32:31.842555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:18.815424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":35,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839094+0000","last_change":"2024-09-19T07:31:32.532426+0000","last_active":"2024-09-19T07:32:31.839094+0000","last_peered":"2024-09-19T07:32:31.839094+0000","last_clean":"2024-09-19T07:32:31.839094+0000","last_became_active":"2024-09-19T07:31:32.532034+0000","last_became_peered":"2024-09-19T07:31:32.532034+0000","last_unstale":"2024-09-19T07:32:31.839094+0000","last_undegraded":"2024-09-19T07:32:31.839094+0000","last_fullsized":"2024-09-19T07:32:31.839094+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:09.421369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836541+0000","last_change":"2024-09-19T07:31:02.444240+0000","last_active":"2024-09-19T07:32:31.836541+0000","last_peered":"2024-09-19T07:32:31.836541+0000","last_clean":"2024-09-19T07:32:31.836541+0000","last_became_active":"2024-09-19T07:31:02.443808+0000","last_became_peered":"2024-09-19T07:31:02.443808+0000","last_unstale":"2024-09-19T07:32:31.836541+0000","last_undegraded":"2024-09-19T07:32:31.836541+0000","last_fullsized":"2024-09-19T07:32:31.836541+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:12.898708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.852906+0000","last_change":"2024-09-19T07:32:31.852388+0000","last_active":"2024-09-19T07:32:31.852906+0000","last_peered":"2024-09-19T07:32:31.852906+0000","last_clean":"2024-09-19T07:32:31.852906+0000","last_became_active":"2024-09-19T07:32:31.843592+0000","last_became_peered":"2024-09-19T07:32:31.843592+0000","last_unstale":"2024-09-19T07:32:31.852906+0000","last_undegraded":"2024-09-19T07:32:31.852906+0000","last_fullsized":"2024-09-19T07:32:31.852906+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:03.020981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.837202+0000","last_change":"2024-09-19T07:31:32.248185+0000","last_active":"2024-09-19T07:32:31.837202+0000","last_peered":"2024-09-19T07:32:31.837202+0000","last_clean":"2024-09-19T07:32:31.837202+0000","last_became_active":"2024-09-19T07:31:32.247514+0000","last_became_peered":"2024-09-19T07:31:32.247514+0000","last_unstale":"2024-09-19T07:32:31.837202+0000","last_undegraded":"2024-09-19T07:32:31.837202+0000","last_fullsized":"2024-09-19T07:32:31.837202+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:09.713374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.844092+0000","last_change":"2024-09-19T07:31:02.421038+0000","last_active":"2024-09-19T07:32:31.844092+0000","last_peered":"2024-09-19T07:32:31.844092+0000","last_clean":"2024-09-19T07:32:31.844092+0000","last_became_active":"2024-09-19T07:31:02.420715+0000","last_became_peered":"2024-09-19T07:31:02.420715+0000","last_unstale":"2024-09-19T07:32:31.844092+0000","last_undegraded":"2024-09-19T07:32:31.844092+0000","last_fullsized":"2024-09-19T07:32:31.844092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:39.943123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"98'31","reported_seq":209,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:54.991983+0000","last_change":"2024-09-19T07:30:59.414095+0000","last_active":"2024-09-19T07:32:54.991983+0000","last_peered":"2024-09-19T07:32:54.991983+0000","last_clean":"2024-09-19T07:32:54.991983+0000","last_became_active":"2024-09-19T07:30:59.413431+0000","last_became_peered":"2024-09-19T07:30:59.413431+0000","last_unstale":"2024-09-19T07:32:54.991983+0000","last_undegraded":"2024-09-19T07:32:54.991983+0000","last_fullsized":"2024-09-19T07:32:54.991983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:00.573369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61523,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":135,"num_read_kb":127,"num_write":32,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185929+0000","last_change":"2024-09-19T07:31:12.500563+0000","last_active":"2024-09-19T07:32:32.185929+0000","last_peered":"2024-09-19T07:32:32.185929+0000","last_clean":"2024-09-19T07:32:32.185929+0000","last_became_active":"2024-09-19T07:31:12.500170+0000","last_became_peered":"2024-09-19T07:31:12.500170+0000","last_unstale":"2024-09-19T07:32:32.185929+0000","last_undegraded":"2024-09-19T07:32:32.185929+0000","last_fullsized":"2024-09-19T07:32:32.185929+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:17.897055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185467+0000","last_change":"2024-09-19T07:31:02.431931+0000","last_active":"2024-09-19T07:32:32.185467+0000","last_peered":"2024-09-19T07:32:32.185467+0000","last_clean":"2024-09-19T07:32:32.185467+0000","last_became_active":"2024-09-19T07:31:02.431579+0000","last_became_peered":"2024-09-19T07:31:02.431579+0000","last_unstale":"2024-09-19T07:32:32.185467+0000","last_undegraded":"2024-09-19T07:32:32.185467+0000","last_fullsized":"2024-09-19T07:32:32.185467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:18.260616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843114+0000","last_change":"2024-09-19T07:30:59.445150+0000","last_active":"2024-09-19T07:32:31.843114+0000","last_peered":"2024-09-19T07:32:31.843114+0000","last_clean":"2024-09-19T07:32:31.843114+0000","last_became_active":"2024-09-19T07:30:59.444776+0000","last_became_peered":"2024-09-19T07:30:59.444776+0000","last_unstale":"2024-09-19T07:32:31.843114+0000","last_undegraded":"2024-09-19T07:32:31.843114+0000","last_fullsized":"2024-09-19T07:32:31.843114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:30.654652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843420+0000","last_change":"2024-09-19T07:31:32.248246+0000","last_active":"2024-09-19T07:32:31.843420+0000","last_peered":"2024-09-19T07:32:31.843420+0000","last_clean":"2024-09-19T07:32:31.843420+0000","last_became_active":"2024-09-19T07:31:32.247456+0000","last_became_peered":"2024-09-19T07:31:32.247456+0000","last_unstale":"2024-09-19T07:32:31.843420+0000","last_undegraded":"2024-09-19T07:32:31.843420+0000","last_fullsized":"2024-09-19T07:32:31.843420+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:31:44.320026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186054+0000","last_change":"2024-09-19T07:31:02.441924+0000","last_active":"2024-09-19T07:32:32.186054+0000","last_peered":"2024-09-19T07:32:32.186054+0000","last_clean":"2024-09-19T07:32:32.186054+0000","last_became_active":"2024-09-19T07:31:02.438247+0000","last_became_peered":"2024-09-19T07:31:02.438247+0000","last_unstale":"2024-09-19T07:32:32.186054+0000","last_undegraded":"2024-09-19T07:32:32.186054+0000","last_fullsized":"2024-09-19T07:32:32.186054+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:12.878185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840450+0000","last_change":"2024-09-19T07:30:59.409679+0000","last_active":"2024-09-19T07:32:31.840450+0000","last_peered":"2024-09-19T07:32:31.840450+0000","last_clean":"2024-09-19T07:32:31.840450+0000","last_became_active":"2024-09-19T07:30:59.409313+0000","last_became_peered":"2024-09-19T07:30:59.409313+0000","last_unstale":"2024-09-19T07:32:31.840450+0000","last_undegraded":"2024-09-19T07:32:31.840450+0000","last_fullsized":"2024-09-19T07:32:31.840450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:42.121358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185477+0000","last_change":"2024-09-19T07:31:12.506596+0000","last_active":"2024-09-19T07:32:32.185477+0000","last_peered":"2024-09-19T07:32:32.185477+0000","last_clean":"2024-09-19T07:32:32.185477+0000","last_became_active":"2024-09-19T07:31:12.505107+0000","last_became_peered":"2024-09-19T07:31:12.505107+0000","last_unstale":"2024-09-19T07:32:32.185477+0000","last_undegraded":"2024-09-19T07:32:32.185477+0000","last_fullsized":"2024-09-19T07:32:32.185477+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:39.157988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185730+0000","last_change":"2024-09-19T07:31:02.429883+0000","last_active":"2024-09-19T07:32:32.185730+0000","last_peered":"2024-09-19T07:32:32.185730+0000","last_clean":"2024-09-19T07:32:32.185730+0000","last_became_active":"2024-09-19T07:31:02.429554+0000","last_became_peered":"2024-09-19T07:31:02.429554+0000","last_unstale":"2024-09-19T07:32:32.185730+0000","last_undegraded":"2024-09-19T07:32:32.185730+0000","last_fullsized":"2024-09-19T07:32:32.185730+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:00.336506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":110,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:53.990638+0000","last_change":"2024-09-19T07:30:59.414735+0000","last_active":"2024-09-19T07:32:53.990638+0000","last_peered":"2024-09-19T07:32:53.990638+0000","last_clean":"2024-09-19T07:32:53.990638+0000","last_became_active":"2024-09-19T07:30:59.414385+0000","last_became_peered":"2024-09-19T07:30:59.414385+0000","last_unstale":"2024-09-19T07:32:53.990638+0000","last_undegraded":"2024-09-19T07:32:53.990638+0000","last_fullsized":"2024-09-19T07:32:53.990638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:43.668794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841496+0000","last_change":"2024-09-19T07:31:12.497242+0000","last_active":"2024-09-19T07:32:31.841496+0000","last_peered":"2024-09-19T07:32:31.841496+0000","last_clean":"2024-09-19T07:32:31.841496+0000","last_became_active":"2024-09-19T07:31:12.496857+0000","last_became_peered":"2024-09-19T07:31:12.496857+0000","last_unstale":"2024-09-19T07:32:31.841496+0000","last_undegraded":"2024-09-19T07:32:31.841496+0000","last_fullsized":"2024-09-19T07:32:31.841496+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:18.418594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185945+0000","last_change":"2024-09-19T07:31:12.504507+0000","last_active":"2024-09-19T07:32:32.185945+0000","last_peered":"2024-09-19T07:32:32.185945+0000","last_clean":"2024-09-19T07:32:32.185945+0000","last_became_active":"2024-09-19T07:31:12.503289+0000","last_became_peered":"2024-09-19T07:31:12.503289+0000","last_unstale":"2024-09-19T07:32:32.185945+0000","last_undegraded":"2024-09-19T07:32:32.185945+0000","last_fullsized":"2024-09-19T07:32:32.185945+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:21.771692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185564+0000","last_change":"2024-09-19T07:31:02.429634+0000","last_active":"2024-09-19T07:32:32.185564+0000","last_peered":"2024-09-19T07:32:32.185564+0000","last_clean":"2024-09-19T07:32:32.185564+0000","last_became_active":"2024-09-19T07:31:02.429280+0000","last_became_peered":"2024-09-19T07:31:02.429280+0000","last_unstale":"2024-09-19T07:32:32.185564+0000","last_undegraded":"2024-09-19T07:32:32.185564+0000","last_fullsized":"2024-09-19T07:32:32.185564+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:13.872594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842927+0000","last_change":"2024-09-19T07:30:59.409961+0000","last_active":"2024-09-19T07:32:31.842927+0000","last_peered":"2024-09-19T07:32:31.842927+0000","last_clean":"2024-09-19T07:32:31.842927+0000","last_became_active":"2024-09-19T07:30:59.409582+0000","last_became_peered":"2024-09-19T07:30:59.409582+0000","last_unstale":"2024-09-19T07:32:31.842927+0000","last_undegraded":"2024-09-19T07:32:31.842927+0000","last_fullsized":"2024-09-19T07:32:31.842927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:31.075378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839571+0000","last_change":"2024-09-19T07:31:12.510999+0000","last_active":"2024-09-19T07:32:31.839571+0000","last_peered":"2024-09-19T07:32:31.839571+0000","last_clean":"2024-09-19T07:32:31.839571+0000","last_became_active":"2024-09-19T07:31:12.510604+0000","last_became_peered":"2024-09-19T07:31:12.510604+0000","last_unstale":"2024-09-19T07:32:31.839571+0000","last_undegraded":"2024-09-19T07:32:31.839571+0000","last_fullsized":"2024-09-19T07:32:31.839571+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:04.779289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841488+0000","last_change":"2024-09-19T07:31:02.428108+0000","last_active":"2024-09-19T07:32:31.841488+0000","last_peered":"2024-09-19T07:32:31.841488+0000","last_clean":"2024-09-19T07:32:31.841488+0000","last_became_active":"2024-09-19T07:31:02.425537+0000","last_became_peered":"2024-09-19T07:31:02.425537+0000","last_unstale":"2024-09-19T07:32:31.841488+0000","last_undegraded":"2024-09-19T07:32:31.841488+0000","last_fullsized":"2024-09-19T07:32:31.841488+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:36.094089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836205+0000","last_change":"2024-09-19T07:30:59.404687+0000","last_active":"2024-09-19T07:32:31.836205+0000","last_peered":"2024-09-19T07:32:31.836205+0000","last_clean":"2024-09-19T07:32:31.836205+0000","last_became_active":"2024-09-19T07:30:59.404222+0000","last_became_peered":"2024-09-19T07:30:59.404222+0000","last_unstale":"2024-09-19T07:32:31.836205+0000","last_undegraded":"2024-09-19T07:32:31.836205+0000","last_fullsized":"2024-09-19T07:32:31.836205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:11.287464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.837093+0000","last_change":"2024-09-19T07:30:59.414711+0000","last_active":"2024-09-19T07:32:31.837093+0000","last_peered":"2024-09-19T07:32:31.837093+0000","last_clean":"2024-09-19T07:32:31.837093+0000","last_became_active":"2024-09-19T07:30:59.414349+0000","last_became_peered":"2024-09-19T07:30:59.414349+0000","last_unstale":"2024-09-19T07:32:31.837093+0000","last_undegraded":"2024-09-19T07:32:31.837093+0000","last_fullsized":"2024-09-19T07:32:31.837093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:57:09.910328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186040+0000","last_change":"2024-09-19T07:31:31.810186+0000","last_active":"2024-09-19T07:32:32.186040+0000","last_peered":"2024-09-19T07:32:32.186040+0000","last_clean":"2024-09-19T07:32:32.186040+0000","last_became_active":"2024-09-19T07:31:31.809861+0000","last_became_peered":"2024-09-19T07:31:31.809861+0000","last_unstale":"2024-09-19T07:32:32.186040+0000","last_undegraded":"2024-09-19T07:32:32.186040+0000","last_fullsized":"2024-09-19T07:32:32.186040+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:49.443012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839492+0000","last_change":"2024-09-19T07:31:12.506060+0000","last_active":"2024-09-19T07:32:31.839492+0000","last_peered":"2024-09-19T07:32:31.839492+0000","last_clean":"2024-09-19T07:32:31.839492+0000","last_became_active":"2024-09-19T07:31:12.505535+0000","last_became_peered":"2024-09-19T07:31:12.505535+0000","last_unstale":"2024-09-19T07:32:31.839492+0000","last_undegraded":"2024-09-19T07:32:31.839492+0000","last_fullsized":"2024-09-19T07:32:31.839492+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:02.820475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'6","reported_seq":116,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:53.991279+0000","last_change":"2024-09-19T07:30:59.408594+0000","last_active":"2024-09-19T07:32:53.991279+0000","last_peered":"2024-09-19T07:32:53.991279+0000","last_clean":"2024-09-19T07:32:53.991279+0000","last_became_active":"2024-09-19T07:30:59.408164+0000","last_became_peered":"2024-09-19T07:30:59.408164+0000","last_unstale":"2024-09-19T07:32:53.991279+0000","last_undegraded":"2024-09-19T07:32:53.991279+0000","last_fullsized":"2024-09-19T07:32:53.991279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:30:58.389296+0000","last_clean_scrub_stamp":"2024-09-19T07:30:58.389296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:23.950637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843141+0000","last_change":"2024-09-19T07:31:02.419940+0000","last_active":"2024-09-19T07:32:31.843141+0000","last_peered":"2024-09-19T07:32:31.843141+0000","last_clean":"2024-09-19T07:32:31.843141+0000","last_became_active":"2024-09-19T07:31:02.419618+0000","last_became_peered":"2024-09-19T07:31:02.419618+0000","last_unstale":"2024-09-19T07:32:31.843141+0000","last_undegraded":"2024-09-19T07:32:31.843141+0000","last_fullsized":"2024-09-19T07:32:31.843141+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:18.208200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840292+0000","last_change":"2024-09-19T07:31:12.497271+0000","last_active":"2024-09-19T07:32:31.840292+0000","last_peered":"2024-09-19T07:32:31.840292+0000","last_clean":"2024-09-19T07:32:31.840292+0000","last_became_active":"2024-09-19T07:31:12.496882+0000","last_became_peered":"2024-09-19T07:31:12.496882+0000","last_unstale":"2024-09-19T07:32:31.840292+0000","last_undegraded":"2024-09-19T07:32:31.840292+0000","last_fullsized":"2024-09-19T07:32:31.840292+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:11.470153+0000","last_clean_scrub_stamp":"2024-09-19T07:31:11.470153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:48.714941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839009+0000","last_change":"2024-09-19T07:31:02.429917+0000","last_active":"2024-09-19T07:32:31.839009+0000","last_peered":"2024-09-19T07:32:31.839009+0000","last_clean":"2024-09-19T07:32:31.839009+0000","last_became_active":"2024-09-19T07:31:02.429535+0000","last_became_peered":"2024-09-19T07:31:02.429535+0000","last_unstale":"2024-09-19T07:32:31.839009+0000","last_undegraded":"2024-09-19T07:32:31.839009+0000","last_fullsized":"2024-09-19T07:32:31.839009+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:21.958491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839931+0000","last_change":"2024-09-19T07:31:02.433135+0000","last_active":"2024-09-19T07:32:31.839931+0000","last_peered":"2024-09-19T07:32:31.839931+0000","last_clean":"2024-09-19T07:32:31.839931+0000","last_became_active":"2024-09-19T07:31:02.432800+0000","last_became_peered":"2024-09-19T07:31:02.432800+0000","last_unstale":"2024-09-19T07:32:31.839931+0000","last_undegraded":"2024-09-19T07:32:31.839931+0000","last_fullsized":"2024-09-19T07:32:31.839931+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:01.559565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839603+0000","last_change":"2024-09-19T07:31:02.438670+0000","last_active":"2024-09-19T07:32:31.839603+0000","last_peered":"2024-09-19T07:32:31.839603+0000","last_clean":"2024-09-19T07:32:31.839603+0000","last_became_active":"2024-09-19T07:31:02.438191+0000","last_became_peered":"2024-09-19T07:31:02.438191+0000","last_unstale":"2024-09-19T07:32:31.839603+0000","last_undegraded":"2024-09-19T07:32:31.839603+0000","last_fullsized":"2024-09-19T07:32:31.839603+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:55.317996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185820+0000","last_change":"2024-09-19T07:31:02.434826+0000","last_active":"2024-09-19T07:32:32.185820+0000","last_peered":"2024-09-19T07:32:32.185820+0000","last_clean":"2024-09-19T07:32:32.185820+0000","last_became_active":"2024-09-19T07:31:02.433695+0000","last_became_peered":"2024-09-19T07:31:02.433695+0000","last_unstale":"2024-09-19T07:32:32.185820+0000","last_undegraded":"2024-09-19T07:32:32.185820+0000","last_fullsized":"2024-09-19T07:32:32.185820+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:33:28.866345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836527+0000","last_change":"2024-09-19T07:31:32.021091+0000","last_active":"2024-09-19T07:32:31.836527+0000","last_peered":"2024-09-19T07:32:31.836527+0000","last_clean":"2024-09-19T07:32:31.836527+0000","last_became_active":"2024-09-19T07:31:32.020537+0000","last_became_peered":"2024-09-19T07:31:32.020537+0000","last_unstale":"2024-09-19T07:32:31.836527+0000","last_undegraded":"2024-09-19T07:32:31.836527+0000","last_fullsized":"2024-09-19T07:32:31.836527+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:31.649790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839411+0000","last_change":"2024-09-19T07:31:02.424514+0000","last_active":"2024-09-19T07:32:31.839411+0000","last_peered":"2024-09-19T07:32:31.839411+0000","last_clean":"2024-09-19T07:32:31.839411+0000","last_became_active":"2024-09-19T07:31:02.424194+0000","last_became_peered":"2024-09-19T07:31:02.424194+0000","last_unstale":"2024-09-19T07:32:31.839411+0000","last_undegraded":"2024-09-19T07:32:31.839411+0000","last_fullsized":"2024-09-19T07:32:31.839411+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:41:47.039372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841312+0000","last_change":"2024-09-19T07:31:02.429624+0000","last_active":"2024-09-19T07:32:31.841312+0000","last_peered":"2024-09-19T07:32:31.841312+0000","last_clean":"2024-09-19T07:32:31.841312+0000","last_became_active":"2024-09-19T07:31:02.429316+0000","last_became_peered":"2024-09-19T07:31:02.429316+0000","last_unstale":"2024-09-19T07:32:31.841312+0000","last_undegraded":"2024-09-19T07:32:31.841312+0000","last_fullsized":"2024-09-19T07:32:31.841312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:15.736433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186389+0000","last_change":"2024-09-19T07:31:31.810511+0000","last_active":"2024-09-19T07:32:32.186389+0000","last_peered":"2024-09-19T07:32:32.186389+0000","last_clean":"2024-09-19T07:32:32.186389+0000","last_became_active":"2024-09-19T07:31:31.809805+0000","last_became_peered":"2024-09-19T07:31:31.809805+0000","last_unstale":"2024-09-19T07:32:32.186389+0000","last_undegraded":"2024-09-19T07:32:32.186389+0000","last_fullsized":"2024-09-19T07:32:32.186389+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:48.457693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842170+0000","last_change":"2024-09-19T07:31:02.433859+0000","last_active":"2024-09-19T07:32:31.842170+0000","last_peered":"2024-09-19T07:32:31.842170+0000","last_clean":"2024-09-19T07:32:31.842170+0000","last_became_active":"2024-09-19T07:31:02.433547+0000","last_became_peered":"2024-09-19T07:31:02.433547+0000","last_unstale":"2024-09-19T07:32:31.842170+0000","last_undegraded":"2024-09-19T07:32:31.842170+0000","last_fullsized":"2024-09-19T07:32:31.842170+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:09.623326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841702+0000","last_change":"2024-09-19T07:31:02.442174+0000","last_active":"2024-09-19T07:32:31.841702+0000","last_peered":"2024-09-19T07:32:31.841702+0000","last_clean":"2024-09-19T07:32:31.841702+0000","last_became_active":"2024-09-19T07:31:02.441851+0000","last_became_peered":"2024-09-19T07:31:02.441851+0000","last_unstale":"2024-09-19T07:32:31.841702+0000","last_undegraded":"2024-09-19T07:32:31.841702+0000","last_fullsized":"2024-09-19T07:32:31.841702+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:59.235498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185362+0000","last_change":"2024-09-19T07:31:02.439526+0000","last_active":"2024-09-19T07:32:32.185362+0000","last_peered":"2024-09-19T07:32:32.185362+0000","last_clean":"2024-09-19T07:32:32.185362+0000","last_became_active":"2024-09-19T07:31:02.438996+0000","last_became_peered":"2024-09-19T07:31:02.438996+0000","last_unstale":"2024-09-19T07:32:32.185362+0000","last_undegraded":"2024-09-19T07:32:32.185362+0000","last_fullsized":"2024-09-19T07:32:32.185362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:55.909727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836389+0000","last_change":"2024-09-19T07:31:02.427770+0000","last_active":"2024-09-19T07:32:31.836389+0000","last_peered":"2024-09-19T07:32:31.836389+0000","last_clean":"2024-09-19T07:32:31.836389+0000","last_became_active":"2024-09-19T07:31:02.427208+0000","last_became_peered":"2024-09-19T07:31:02.427208+0000","last_unstale":"2024-09-19T07:32:31.836389+0000","last_undegraded":"2024-09-19T07:32:31.836389+0000","last_fullsized":"2024-09-19T07:32:31.836389+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:01.171781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839160+0000","last_change":"2024-09-19T07:31:02.433089+0000","last_active":"2024-09-19T07:32:31.839160+0000","last_peered":"2024-09-19T07:32:31.839160+0000","last_clean":"2024-09-19T07:32:31.839160+0000","last_became_active":"2024-09-19T07:31:02.432744+0000","last_became_peered":"2024-09-19T07:31:02.432744+0000","last_unstale":"2024-09-19T07:32:31.839160+0000","last_undegraded":"2024-09-19T07:32:31.839160+0000","last_fullsized":"2024-09-19T07:32:31.839160+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:16:32.959555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840486+0000","last_change":"2024-09-19T07:31:02.435482+0000","last_active":"2024-09-19T07:32:31.840486+0000","last_peered":"2024-09-19T07:32:31.840486+0000","last_clean":"2024-09-19T07:32:31.840486+0000","last_became_active":"2024-09-19T07:31:02.435164+0000","last_became_peered":"2024-09-19T07:31:02.435164+0000","last_unstale":"2024-09-19T07:32:31.840486+0000","last_undegraded":"2024-09-19T07:32:31.840486+0000","last_fullsized":"2024-09-19T07:32:31.840486+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:54.478049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186419+0000","last_change":"2024-09-19T07:31:02.426768+0000","last_active":"2024-09-19T07:32:32.186419+0000","last_peered":"2024-09-19T07:32:32.186419+0000","last_clean":"2024-09-19T07:32:32.186419+0000","last_became_active":"2024-09-19T07:31:02.425592+0000","last_became_peered":"2024-09-19T07:31:02.425592+0000","last_unstale":"2024-09-19T07:32:32.186419+0000","last_undegraded":"2024-09-19T07:32:32.186419+0000","last_fullsized":"2024-09-19T07:32:32.186419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:44.985532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840446+0000","last_change":"2024-09-19T07:31:02.426233+0000","last_active":"2024-09-19T07:32:31.840446+0000","last_peered":"2024-09-19T07:32:31.840446+0000","last_clean":"2024-09-19T07:32:31.840446+0000","last_became_active":"2024-09-19T07:31:02.425777+0000","last_became_peered":"2024-09-19T07:31:02.425777+0000","last_unstale":"2024-09-19T07:32:31.840446+0000","last_undegraded":"2024-09-19T07:32:31.840446+0000","last_fullsized":"2024-09-19T07:32:31.840446+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:31.497641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185109+0000","last_change":"2024-09-19T07:31:02.428671+0000","last_active":"2024-09-19T07:32:32.185109+0000","last_peered":"2024-09-19T07:32:32.185109+0000","last_clean":"2024-09-19T07:32:32.185109+0000","last_became_active":"2024-09-19T07:31:02.425930+0000","last_became_peered":"2024-09-19T07:31:02.425930+0000","last_unstale":"2024-09-19T07:32:32.185109+0000","last_undegraded":"2024-09-19T07:32:32.185109+0000","last_fullsized":"2024-09-19T07:32:32.185109+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:21.721694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185416+0000","last_change":"2024-09-19T07:31:02.441450+0000","last_active":"2024-09-19T07:32:32.185416+0000","last_peered":"2024-09-19T07:32:32.185416+0000","last_clean":"2024-09-19T07:32:32.185416+0000","last_became_active":"2024-09-19T07:31:02.436673+0000","last_became_peered":"2024-09-19T07:31:02.436673+0000","last_unstale":"2024-09-19T07:32:32.185416+0000","last_undegraded":"2024-09-19T07:32:32.185416+0000","last_fullsized":"2024-09-19T07:32:32.185416+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:29.744665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185613+0000","last_change":"2024-09-19T07:31:02.427449+0000","last_active":"2024-09-19T07:32:32.185613+0000","last_peered":"2024-09-19T07:32:32.185613+0000","last_clean":"2024-09-19T07:32:32.185613+0000","last_became_active":"2024-09-19T07:31:02.426601+0000","last_became_peered":"2024-09-19T07:31:02.426601+0000","last_unstale":"2024-09-19T07:32:32.185613+0000","last_undegraded":"2024-09-19T07:32:32.185613+0000","last_fullsized":"2024-09-19T07:32:32.185613+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:36.752529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.836388+0000","last_change":"2024-09-19T07:31:02.424816+0000","last_active":"2024-09-19T07:32:31.836388+0000","last_peered":"2024-09-19T07:32:31.836388+0000","last_clean":"2024-09-19T07:32:31.836388+0000","last_became_active":"2024-09-19T07:31:02.423729+0000","last_became_peered":"2024-09-19T07:31:02.423729+0000","last_unstale":"2024-09-19T07:32:31.836388+0000","last_undegraded":"2024-09-19T07:32:31.836388+0000","last_fullsized":"2024-09-19T07:32:31.836388+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:39.546052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843223+0000","last_change":"2024-09-19T07:31:02.428444+0000","last_active":"2024-09-19T07:32:31.843223+0000","last_peered":"2024-09-19T07:32:31.843223+0000","last_clean":"2024-09-19T07:32:31.843223+0000","last_became_active":"2024-09-19T07:31:02.428121+0000","last_became_peered":"2024-09-19T07:31:02.428121+0000","last_unstale":"2024-09-19T07:32:31.843223+0000","last_undegraded":"2024-09-19T07:32:31.843223+0000","last_fullsized":"2024-09-19T07:32:31.843223+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:52.649412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840453+0000","last_change":"2024-09-19T07:31:02.423102+0000","last_active":"2024-09-19T07:32:31.840453+0000","last_peered":"2024-09-19T07:32:31.840453+0000","last_clean":"2024-09-19T07:32:31.840453+0000","last_became_active":"2024-09-19T07:31:02.422672+0000","last_became_peered":"2024-09-19T07:31:02.422672+0000","last_unstale":"2024-09-19T07:32:31.840453+0000","last_undegraded":"2024-09-19T07:32:31.840453+0000","last_fullsized":"2024-09-19T07:32:31.840453+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:45.733320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.837494+0000","last_change":"2024-09-19T07:31:02.434599+0000","last_active":"2024-09-19T07:32:31.837494+0000","last_peered":"2024-09-19T07:32:31.837494+0000","last_clean":"2024-09-19T07:32:31.837494+0000","last_became_active":"2024-09-19T07:31:02.434263+0000","last_became_peered":"2024-09-19T07:31:02.434263+0000","last_unstale":"2024-09-19T07:32:31.837494+0000","last_undegraded":"2024-09-19T07:32:31.837494+0000","last_fullsized":"2024-09-19T07:32:31.837494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:16.104483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185751+0000","last_change":"2024-09-19T07:31:02.429278+0000","last_active":"2024-09-19T07:32:32.185751+0000","last_peered":"2024-09-19T07:32:32.185751+0000","last_clean":"2024-09-19T07:32:32.185751+0000","last_became_active":"2024-09-19T07:31:02.428991+0000","last_became_peered":"2024-09-19T07:31:02.428991+0000","last_unstale":"2024-09-19T07:32:32.185751+0000","last_undegraded":"2024-09-19T07:32:32.185751+0000","last_fullsized":"2024-09-19T07:32:32.185751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:02.421684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.841490+0000","last_change":"2024-09-19T07:31:02.436412+0000","last_active":"2024-09-19T07:32:31.841490+0000","last_peered":"2024-09-19T07:32:31.841490+0000","last_clean":"2024-09-19T07:32:31.841490+0000","last_became_active":"2024-09-19T07:31:02.436101+0000","last_became_peered":"2024-09-19T07:31:02.436101+0000","last_unstale":"2024-09-19T07:32:31.841490+0000","last_undegraded":"2024-09-19T07:32:31.841490+0000","last_fullsized":"2024-09-19T07:32:31.841490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:34.827734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.839956+0000","last_change":"2024-09-19T07:31:02.423414+0000","last_active":"2024-09-19T07:32:31.839956+0000","last_peered":"2024-09-19T07:32:31.839956+0000","last_clean":"2024-09-19T07:32:31.839956+0000","last_became_active":"2024-09-19T07:31:02.423100+0000","last_became_peered":"2024-09-19T07:31:02.423100+0000","last_unstale":"2024-09-19T07:32:31.839956+0000","last_undegraded":"2024-09-19T07:32:31.839956+0000","last_fullsized":"2024-09-19T07:32:31.839956+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:15.269095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.837049+0000","last_change":"2024-09-19T07:31:02.434953+0000","last_active":"2024-09-19T07:32:31.837049+0000","last_peered":"2024-09-19T07:32:31.837049+0000","last_clean":"2024-09-19T07:32:31.837049+0000","last_became_active":"2024-09-19T07:31:02.434662+0000","last_became_peered":"2024-09-19T07:31:02.434662+0000","last_unstale":"2024-09-19T07:32:31.837049+0000","last_undegraded":"2024-09-19T07:32:31.837049+0000","last_fullsized":"2024-09-19T07:32:31.837049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:17.132108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.842883+0000","last_change":"2024-09-19T07:31:02.427959+0000","last_active":"2024-09-19T07:32:31.842883+0000","last_peered":"2024-09-19T07:32:31.842883+0000","last_clean":"2024-09-19T07:32:31.842883+0000","last_became_active":"2024-09-19T07:31:02.427671+0000","last_became_peered":"2024-09-19T07:31:02.427671+0000","last_unstale":"2024-09-19T07:32:31.842883+0000","last_undegraded":"2024-09-19T07:32:31.842883+0000","last_fullsized":"2024-09-19T07:32:31.842883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:19.608005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.843150+0000","last_change":"2024-09-19T07:31:02.433310+0000","last_active":"2024-09-19T07:32:31.843150+0000","last_peered":"2024-09-19T07:32:31.843150+0000","last_clean":"2024-09-19T07:32:31.843150+0000","last_became_active":"2024-09-19T07:31:02.432922+0000","last_became_peered":"2024-09-19T07:31:02.432922+0000","last_unstale":"2024-09-19T07:32:31.843150+0000","last_undegraded":"2024-09-19T07:32:31.843150+0000","last_fullsized":"2024-09-19T07:32:31.843150+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:30.043075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.185720+0000","last_change":"2024-09-19T07:31:02.429452+0000","last_active":"2024-09-19T07:32:32.185720+0000","last_peered":"2024-09-19T07:32:32.185720+0000","last_clean":"2024-09-19T07:32:32.185720+0000","last_became_active":"2024-09-19T07:31:02.429142+0000","last_became_peered":"2024-09-19T07:31:02.429142+0000","last_unstale":"2024-09-19T07:32:32.185720+0000","last_undegraded":"2024-09-19T07:32:32.185720+0000","last_fullsized":"2024-09-19T07:32:32.185720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:42.107179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:32.186005+0000","last_change":"2024-09-19T07:31:02.431197+0000","last_active":"2024-09-19T07:32:32.186005+0000","last_peered":"2024-09-19T07:32:32.186005+0000","last_clean":"2024-09-19T07:32:32.186005+0000","last_became_active":"2024-09-19T07:31:02.430812+0000","last_became_peered":"2024-09-19T07:31:02.430812+0000","last_unstale":"2024-09-19T07:32:32.186005+0000","last_undegraded":"2024-09-19T07:32:32.186005+0000","last_fullsized":"2024-09-19T07:32:32.186005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:10.682008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-19T07:32:31.840341+0000","last_change":"2024-09-19T07:31:02.426753+0000","last_active":"2024-09-19T07:32:31.840341+0000","last_peered":"2024-09-19T07:32:31.840341+0000","last_clean":"2024-09-19T07:32:31.840341+0000","last_became_active":"2024-09-19T07:31:02.425870+0000","last_became_peered":"2024-09-19T07:31:02.425870+0000","last_unstale":"2024-09-19T07:32:31.840341+0000","last_undegraded":"2024-09-19T07:32:31.840341+0000","last_fullsized":"2024-09-19T07:32:31.840341+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:01.399019+0000","last_clean_scrub_stamp":"2024-09-19T07:31:01.399019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:05.410032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65919,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":413,"num_read_kb":276,"num_write":165,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":197757,"data_compressed":5616,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247636,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351014,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1110016,"data_stored":572114,"data_compressed":32327,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579947,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28240,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716912,"statfs":{"total":95995035648,"available":95966117888,"internally_reserved":0,"allocated":1241088,"data_stored":1221242,"data_compressed":70047,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743474,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1122304,"data_stored":576390,"data_compressed":32327,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005111,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1114112,"data_stored":572192,"data_compressed":32327,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299452,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1167360,"data_stored":1168518,"data_compressed":68175,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462977,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1097728,"data_stored":567470,"data_compressed":32327,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28176,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716976,"statfs":{"total":95995035648,"available":95966183424,"internally_reserved":0,"allocated":1175552,"data_stored":1163713,"data_compressed":68175,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888140,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1110016,"data_stored":572513,"data_compressed":32327,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149777,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1110016,"data_stored":567664,"data_compressed":32327,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28184,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716968,"statfs":{"total":95995035648,"available":95966175232,"internally_reserved":0,"allocated":1179648,"data_stored":630612,"data_compressed":34199,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1159168,"data_stored":628903,"data_compressed":34199,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32288,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712864,"statfs":{"total":95995035648,"available":95961972736,"internally_reserved":0,"allocated":1122304,"data_stored":568858,"data_compressed":32327,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35848,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61523,"data_compressed":1872,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63232,"data_compressed":1872,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63494,"data_compressed":1872,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:33:02.014 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T07:33:02.014 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T07:33:02.014 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T07:33:02.015 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T07:33:02.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:01 smithi203 ceph-mon[35335]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:02.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:01 smithi189 ceph-mon[34851]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:02.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:01 smithi159 ceph-mon[30472]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:02.444 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:02.959 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:02 smithi189 ceph-mon[34851]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:33:02.959 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:02 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:03.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.006+0000 7f2676200640 1 -- 172.21.15.159:0/3895137960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2670106f20 msgr2=0x7f267010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:03.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.006+0000 7f2676200640 1 --2- 172.21.15.159:0/3895137960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2670106f20 0x7f267010eaa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2668009ca0 tx=0x7f266802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:33:03.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.007+0000 7f2676200640 1 -- 172.21.15.159:0/3895137960 shutdown_connections 2024-09-19T07:33:03.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.007+0000 7f2676200640 1 --2- 172.21.15.159:0/3895137960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2670106f20 0x7f267010eaa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:03.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.007+0000 7f2676200640 1 --2- 172.21.15.159:0/3895137960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26701027a0 0x7f26701067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:03.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.007+0000 7f2676200640 1 --2- 172.21.15.159:0/3895137960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2670101df0 0x7f26701021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:03.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.007+0000 7f2676200640 1 -- 172.21.15.159:0/3895137960 >> 172.21.15.159:0/3895137960 conn(0x7f26700fd8b0 msgr2=0x7f26700ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:03.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.007+0000 7f2676200640 1 -- 172.21.15.159:0/3895137960 shutdown_connections 2024-09-19T07:33:03.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.007+0000 7f2676200640 1 -- 172.21.15.159:0/3895137960 wait complete. 2024-09-19T07:33:03.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.008+0000 7f2676200640 1 Processor -- start 2024-09-19T07:33:03.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.008+0000 7f2676200640 1 -- start start 2024-09-19T07:33:03.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.008+0000 7f2676200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2670101df0 0x7f26700744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:03.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.008+0000 7f2676200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26701027a0 0x7f2670074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:03.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.008+0000 7f2676200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2670106f20 0x7f2670074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:03.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.008+0000 7f2676200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2670111fd0 con 0x7f2670106f20 2024-09-19T07:33:03.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.008+0000 7f2676200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2670111e50 con 0x7f26701027a0 2024-09-19T07:33:03.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.008+0000 7f2676200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2670112150 con 0x7f2670101df0 2024-09-19T07:33:03.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.008+0000 7f266f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2670101df0 0x7f26700744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:03.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.008+0000 7f266fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2670106f20 0x7f2670074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:03.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.008+0000 7f266f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2670101df0 0x7f26700744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:56338/0 (socket says 172.21.15.159:56338) 2024-09-19T07:33:03.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.008+0000 7f266f400640 1 -- 172.21.15.159:0/1988193229 learned_addr learned my addr 172.21.15.159:0/1988193229 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:03.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.008+0000 7f266ea00640 1 --2- 172.21.15.159:0/1988193229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26701027a0 0x7f2670074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:03.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.009+0000 7f266f400640 1 -- 172.21.15.159:0/1988193229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26701027a0 msgr2=0x7f2670074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:03.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.009+0000 7f266f400640 1 --2- 172.21.15.159:0/1988193229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26701027a0 0x7f2670074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:03.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.009+0000 7f266f400640 1 -- 172.21.15.159:0/1988193229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2670106f20 msgr2=0x7f2670074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:03.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.009+0000 7f266f400640 1 --2- 172.21.15.159:0/1988193229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2670106f20 0x7f2670074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:03.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.009+0000 7f266f400640 1 -- 172.21.15.159:0/1988193229 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26701a6890 con 0x7f2670101df0 2024-09-19T07:33:03.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.009+0000 7f266f400640 1 --2- 172.21.15.159:0/1988193229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2670101df0 0x7f26700744f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f266000e9f0 tx=0x7f266000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:03.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.009+0000 7f2667e00640 1 -- 172.21.15.159:0/1988193229 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266000cd80 con 0x7f2670101df0 2024-09-19T07:33:03.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.009+0000 7f2667e00640 1 -- 172.21.15.159:0/1988193229 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2660010910 con 0x7f2670101df0 2024-09-19T07:33:03.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.009+0000 7f2667e00640 1 -- 172.21.15.159:0/1988193229 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660018990 con 0x7f2670101df0 2024-09-19T07:33:03.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.009+0000 7f2676200640 1 -- 172.21.15.159:0/1988193229 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26701a6b80 con 0x7f2670101df0 2024-09-19T07:33:03.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.010+0000 7f2676200640 1 -- 172.21.15.159:0/1988193229 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26701a7070 con 0x7f2670101df0 2024-09-19T07:33:03.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.011+0000 7f2676200640 1 -- 172.21.15.159:0/1988193229 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26701021d0 con 0x7f2670101df0 2024-09-19T07:33:03.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.013+0000 7f2667e00640 1 -- 172.21.15.159:0/1988193229 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f2660010ac0 con 0x7f2670101df0 2024-09-19T07:33:03.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.014+0000 7f2667e00640 1 --2- 172.21.15.159:0/1988193229 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f26540a32d0 0x7f26540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:03.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.015+0000 7f2667e00640 1 -- 172.21.15.159:0/1988193229 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f26600d0470 con 0x7f2670101df0 2024-09-19T07:33:03.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.015+0000 7f266ea00640 1 --2- 172.21.15.159:0/1988193229 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f26540a32d0 0x7f26540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:03.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.015+0000 7f266ea00640 1 --2- 172.21.15.159:0/1988193229 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f26540a32d0 0x7f26540a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2670075f80 tx=0x7f265c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:03.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.016+0000 7f2667e00640 1 -- 172.21.15.159:0/1988193229 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f266009c630 con 0x7f2670101df0 2024-09-19T07:33:03.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.239+0000 7f2676200640 1 -- 172.21.15.159:0/1988193229 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f26701a7390 con 0x7f2670101df0 2024-09-19T07:33:03.242 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:02 smithi159 ceph-mon[30472]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:33:03.242 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:02 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:03.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.240+0000 7f2667e00640 1 -- 172.21.15.159:0/1988193229 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f26600a0930 con 0x7f2670101df0 2024-09-19T07:33:03.242 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:33:03.242 INFO:teuthology.orchestra.run.smithi159.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:33:03.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.243+0000 7f2676200640 1 -- 172.21.15.159:0/1988193229 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f26540a32d0 msgr2=0x7f26540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:03.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.243+0000 7f2676200640 1 --2- 172.21.15.159:0/1988193229 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f26540a32d0 0x7f26540a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2670075f80 tx=0x7f265c005f90 comp rx=0 tx=0).stop 2024-09-19T07:33:03.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.243+0000 7f2676200640 1 -- 172.21.15.159:0/1988193229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2670101df0 msgr2=0x7f26700744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:03.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.243+0000 7f2676200640 1 --2- 172.21.15.159:0/1988193229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2670101df0 0x7f26700744f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f266000e9f0 tx=0x7f266000eeb0 comp rx=0 tx=0).stop 2024-09-19T07:33:03.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.243+0000 7f2676200640 1 -- 172.21.15.159:0/1988193229 shutdown_connections 2024-09-19T07:33:03.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.243+0000 7f2676200640 1 --2- 172.21.15.159:0/1988193229 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f26540a32d0 0x7f26540a5790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:03.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.243+0000 7f2676200640 1 --2- 172.21.15.159:0/1988193229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2670106f20 0x7f2670074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:03.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.243+0000 7f2676200640 1 --2- 172.21.15.159:0/1988193229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26701027a0 0x7f2670074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:03.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.243+0000 7f2676200640 1 --2- 172.21.15.159:0/1988193229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2670101df0 0x7f26700744f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:03.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.243+0000 7f2676200640 1 -- 172.21.15.159:0/1988193229 >> 172.21.15.159:0/1988193229 conn(0x7f26700fd8b0 msgr2=0x7f26700fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:03.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.244+0000 7f2676200640 1 -- 172.21.15.159:0/1988193229 shutdown_connections 2024-09-19T07:33:03.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:03.244+0000 7f2676200640 1 -- 172.21.15.159:0/1988193229 wait complete. 2024-09-19T07:33:03.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:02 smithi203 ceph-mon[35335]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:33:03.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:02 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:04.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:03 smithi159 ceph-mon[30472]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:33:04.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:03 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1988193229' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T07:33:04.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:03 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:04.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:03 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:04.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:03 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:04.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:03 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:04.209 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T07:33:04.209 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:33:04.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:03 smithi203 ceph-mon[35335]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:33:04.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:03 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1988193229' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T07:33:04.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:03 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:04.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:03 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:04.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:03 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:04.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:03 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:04.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:03 smithi189 ceph-mon[34851]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:33:04.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:03 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1988193229' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T07:33:04.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:03 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:04.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:03 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:04.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:03 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:04.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:03 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:04.720 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:05.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.324+0000 7fea4d200640 1 -- 172.21.15.159:0/1346549770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea48103ff0 msgr2=0x7fea481043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:05.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.324+0000 7fea4d200640 1 --2- 172.21.15.159:0/1346549770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea48103ff0 0x7fea481043d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fea38009900 tx=0x7fea3802a410 comp rx=0 tx=0).stop 2024-09-19T07:33:05.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.324+0000 7fea4d200640 1 -- 172.21.15.159:0/1346549770 shutdown_connections 2024-09-19T07:33:05.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.324+0000 7fea4d200640 1 --2- 172.21.15.159:0/1346549770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea48109120 0x7fea48110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:05.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.324+0000 7fea4d200640 1 --2- 172.21.15.159:0/1346549770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea481049a0 0x7fea481089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:05.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.324+0000 7fea4d200640 1 --2- 172.21.15.159:0/1346549770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea48103ff0 0x7fea481043d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:05.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.324+0000 7fea4d200640 1 -- 172.21.15.159:0/1346549770 >> 172.21.15.159:0/1346549770 conn(0x7fea480ffa90 msgr2=0x7fea48101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:05.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.324+0000 7fea4d200640 1 -- 172.21.15.159:0/1346549770 shutdown_connections 2024-09-19T07:33:05.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.324+0000 7fea4d200640 1 -- 172.21.15.159:0/1346549770 wait complete. 2024-09-19T07:33:05.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.325+0000 7fea4d200640 1 Processor -- start 2024-09-19T07:33:05.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.325+0000 7fea4d200640 1 -- start start 2024-09-19T07:33:05.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.326+0000 7fea4d200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea48103ff0 0x7fea4819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:05.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.326+0000 7fea4d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea481049a0 0x7fea4819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:05.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.326+0000 7fea4d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea48109120 0x7fea4819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:05.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.326+0000 7fea4d200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea48114170 con 0x7fea481049a0 2024-09-19T07:33:05.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.326+0000 7fea4d200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea48113ff0 con 0x7fea48109120 2024-09-19T07:33:05.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.326+0000 7fea4d200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fea481142f0 con 0x7fea48103ff0 2024-09-19T07:33:05.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.326+0000 7fea45400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea481049a0 0x7fea4819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:05.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.326+0000 7fea45400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea481049a0 0x7fea4819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60948/0 (socket says 172.21.15.159:60948) 2024-09-19T07:33:05.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.326+0000 7fea45e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea48103ff0 0x7fea4819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:05.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.326+0000 7fea45400640 1 -- 172.21.15.159:0/3152773620 learned_addr learned my addr 172.21.15.159:0/3152773620 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:05.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.326+0000 7fea46800640 1 --2- 172.21.15.159:0/3152773620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea48109120 0x7fea4819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:05.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.327+0000 7fea45400640 1 -- 172.21.15.159:0/3152773620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea48103ff0 msgr2=0x7fea4819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:05.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.327+0000 7fea45400640 1 --2- 172.21.15.159:0/3152773620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea48103ff0 0x7fea4819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:05.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.327+0000 7fea45400640 1 -- 172.21.15.159:0/3152773620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea48109120 msgr2=0x7fea4819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:05.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.327+0000 7fea45400640 1 --2- 172.21.15.159:0/3152773620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea48109120 0x7fea4819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:05.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.327+0000 7fea45400640 1 -- 172.21.15.159:0/3152773620 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4819f620 con 0x7fea481049a0 2024-09-19T07:33:05.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.327+0000 7fea45400640 1 --2- 172.21.15.159:0/3152773620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea481049a0 0x7fea4819abb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fea3400d950 tx=0x7fea3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:05.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.327+0000 7fea3ea00640 1 -- 172.21.15.159:0/3152773620 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34017070 con 0x7fea481049a0 2024-09-19T07:33:05.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.327+0000 7fea3ea00640 1 -- 172.21.15.159:0/3152773620 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea34009c30 con 0x7fea481049a0 2024-09-19T07:33:05.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.327+0000 7fea4d200640 1 -- 172.21.15.159:0/3152773620 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4819f910 con 0x7fea481049a0 2024-09-19T07:33:05.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.327+0000 7fea3ea00640 1 -- 172.21.15.159:0/3152773620 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3400f6b0 con 0x7fea481049a0 2024-09-19T07:33:05.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.328+0000 7fea4d200640 1 -- 172.21.15.159:0/3152773620 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea481a71f0 con 0x7fea481049a0 2024-09-19T07:33:05.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.329+0000 7fea4d200640 1 -- 172.21.15.159:0/3152773620 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea481043d0 con 0x7fea481049a0 2024-09-19T07:33:05.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.330+0000 7fea3ea00640 1 -- 172.21.15.159:0/3152773620 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fea340109d0 con 0x7fea481049a0 2024-09-19T07:33:05.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.332+0000 7fea3ea00640 1 --2- 172.21.15.159:0/3152773620 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fea2c0a3350 0x7fea2c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:05.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.332+0000 7fea3ea00640 1 -- 172.21.15.159:0/3152773620 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fea340cf9f0 con 0x7fea481049a0 2024-09-19T07:33:05.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.333+0000 7fea45e00640 1 --2- 172.21.15.159:0/3152773620 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fea2c0a3350 0x7fea2c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:05.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.334+0000 7fea45e00640 1 --2- 172.21.15.159:0/3152773620 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fea2c0a3350 0x7fea2c0a5810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fea38000c00 tx=0x7fea38005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:05.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.337+0000 7fea3ea00640 1 -- 172.21.15.159:0/3152773620 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fea34009de0 con 0x7fea481049a0 2024-09-19T07:33:05.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.586+0000 7fea4d200640 1 -- 172.21.15.159:0/3152773620 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea4810c300 con 0x7fea481049a0 2024-09-19T07:33:05.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.587+0000 7fea3ea00640 1 -- 172.21.15.159:0/3152773620 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11936 (secure 0 0 0) 0x7fea3409bb50 con 0x7fea481049a0 2024-09-19T07:33:05.589 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:33:05.589 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":25,"btime":"2024-09-19T07:31:54:895489+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3464449771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3464449771},{"type":"v1","addr":"172.21.15.159:6837","nonce":3464449771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2036264856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2036264856},{"type":"v1","addr":"172.21.15.159:6839","nonce":2036264856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6841/2874566405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6840","nonce":2874566405},{"type":"v1","addr":"172.21.15.159:6841","nonce":2874566405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:31:03.518840+0000","modified":"2024-09-19T07:31:53.954806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24542,"mds_1":24514,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/1340279549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1340279549},{"type":"v1","addr":"172.21.15.189:6835","nonce":1340279549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/937970878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":937970878},{"type":"v1","addr":"172.21.15.203:6835","nonce":937970878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1053443810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1053443810},{"type":"v1","addr":"172.21.15.203:6837","nonce":1053443810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.189:6837/1030825585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1030825585},{"type":"v1","addr":"172.21.15.189:6837","nonce":1030825585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.203:6839/3862781462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":3862781462},{"type":"v1","addr":"172.21.15.203:6839","nonce":3862781462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.189:6839/2361818663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2361818663},{"type":"v1","addr":"172.21.15.189:6839","nonce":2361818663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24506,24514,24542]},"id":1}]} 2024-09-19T07:33:05.589 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 25 2024-09-19T07:33:05.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.590+0000 7fea4d200640 1 -- 172.21.15.159:0/3152773620 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fea2c0a3350 msgr2=0x7fea2c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:05.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.590+0000 7fea4d200640 1 --2- 172.21.15.159:0/3152773620 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fea2c0a3350 0x7fea2c0a5810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fea38000c00 tx=0x7fea38005b50 comp rx=0 tx=0).stop 2024-09-19T07:33:05.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.590+0000 7fea4d200640 1 -- 172.21.15.159:0/3152773620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea481049a0 msgr2=0x7fea4819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:05.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.590+0000 7fea4d200640 1 --2- 172.21.15.159:0/3152773620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea481049a0 0x7fea4819abb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fea3400d950 tx=0x7fea3400de10 comp rx=0 tx=0).stop 2024-09-19T07:33:05.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.590+0000 7fea4d200640 1 -- 172.21.15.159:0/3152773620 shutdown_connections 2024-09-19T07:33:05.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.590+0000 7fea4d200640 1 --2- 172.21.15.159:0/3152773620 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fea2c0a3350 0x7fea2c0a5810 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:05.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.590+0000 7fea4d200640 1 --2- 172.21.15.159:0/3152773620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea48109120 0x7fea4819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:05.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.590+0000 7fea4d200640 1 --2- 172.21.15.159:0/3152773620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea481049a0 0x7fea4819abb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:05.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.590+0000 7fea4d200640 1 --2- 172.21.15.159:0/3152773620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea48103ff0 0x7fea4819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:05.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.590+0000 7fea4d200640 1 -- 172.21.15.159:0/3152773620 >> 172.21.15.159:0/3152773620 conn(0x7fea480ffa90 msgr2=0x7fea48100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:05.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.591+0000 7fea4d200640 1 -- 172.21.15.159:0/3152773620 shutdown_connections 2024-09-19T07:33:05.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:05.591+0000 7fea4d200640 1 -- 172.21.15.159:0/3152773620 wait complete. 2024-09-19T07:33:06.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:05 smithi203 ceph-mon[35335]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:06.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:05 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3152773620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:33:06.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:05 smithi189 ceph-mon[34851]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:06.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:05 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3152773620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:33:06.349 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:05 smithi159 ceph-mon[30472]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:06.349 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:05 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3152773620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:33:06.388 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:33:06.788 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:07.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.410+0000 7f05cd200640 1 -- 172.21.15.159:0/400113288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c8102370 msgr2=0x7f05c810e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:07.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.410+0000 7f05cd200640 1 --2- 172.21.15.159:0/400113288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c8102370 0x7f05c810e8a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009930 tx=0x7f05c002a420 comp rx=0 tx=0).stop 2024-09-19T07:33:07.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.410+0000 7f05cd200640 1 -- 172.21.15.159:0/400113288 shutdown_connections 2024-09-19T07:33:07.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.410+0000 7f05cd200640 1 --2- 172.21.15.159:0/400113288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c8102370 0x7f05c810e8a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:07.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.410+0000 7f05cd200640 1 --2- 172.21.15.159:0/400113288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05c81019d0 0x7f05c8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:07.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.410+0000 7f05cd200640 1 --2- 172.21.15.159:0/400113288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05c81079d0 0x7f05c8107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:07.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.410+0000 7f05cd200640 1 -- 172.21.15.159:0/400113288 >> 172.21.15.159:0/400113288 conn(0x7f05c80fd710 msgr2=0x7f05c80ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:07.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.411+0000 7f05cd200640 1 -- 172.21.15.159:0/400113288 shutdown_connections 2024-09-19T07:33:07.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.411+0000 7f05cd200640 1 -- 172.21.15.159:0/400113288 wait complete. 2024-09-19T07:33:07.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.412+0000 7f05cd200640 1 Processor -- start 2024-09-19T07:33:07.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.412+0000 7f05cd200640 1 -- start start 2024-09-19T07:33:07.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.412+0000 7f05cd200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05c81019d0 0x7f05c819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:07.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.412+0000 7f05cd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c8102370 0x7f05c819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:07.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.412+0000 7f05cd200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05c81079d0 0x7f05c819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:07.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.412+0000 7f05cd200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f05c8111dd0 con 0x7f05c8102370 2024-09-19T07:33:07.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.412+0000 7f05cd200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05c8111c50 con 0x7f05c81019d0 2024-09-19T07:33:07.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.412+0000 7f05cd200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f05c8111f50 con 0x7f05c81079d0 2024-09-19T07:33:07.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.413+0000 7f05c7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05c81019d0 0x7f05c819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:07.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.413+0000 7f05c7e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05c81079d0 0x7f05c819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:07.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.413+0000 7f05c7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05c81019d0 0x7f05c819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:54508/0 (socket says 172.21.15.159:54508) 2024-09-19T07:33:07.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.413+0000 7f05c7400640 1 -- 172.21.15.159:0/2968989516 learned_addr learned my addr 172.21.15.159:0/2968989516 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:07.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.413+0000 7f05c7400640 1 -- 172.21.15.159:0/2968989516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05c81079d0 msgr2=0x7f05c819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:07.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.413+0000 7f05c7400640 1 --2- 172.21.15.159:0/2968989516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05c81079d0 0x7f05c819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:07.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.413+0000 7f05c7400640 1 -- 172.21.15.159:0/2968989516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c8102370 msgr2=0x7f05c819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:07.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.413+0000 7f05c7400640 1 --2- 172.21.15.159:0/2968989516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c8102370 0x7f05c819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:07.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.413+0000 7f05c7400640 1 -- 172.21.15.159:0/2968989516 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c819f420 con 0x7f05c81019d0 2024-09-19T07:33:07.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.413+0000 7f05c7e00640 1 --2- 172.21.15.159:0/2968989516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05c81079d0 0x7f05c819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:07.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.413+0000 7f05c7400640 1 --2- 172.21.15.159:0/2968989516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05c81019d0 0x7f05c819a470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f05b800eb90 tx=0x7f05b800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:07.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.414+0000 7f05c4c00640 1 -- 172.21.15.159:0/2968989516 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8004d20 con 0x7f05c81019d0 2024-09-19T07:33:07.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.414+0000 7f05cd200640 1 -- 172.21.15.159:0/2968989516 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c819f710 con 0x7f05c81019d0 2024-09-19T07:33:07.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.414+0000 7f05c4c00640 1 -- 172.21.15.159:0/2968989516 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b800ee90 con 0x7f05c81019d0 2024-09-19T07:33:07.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.414+0000 7f05c4c00640 1 -- 172.21.15.159:0/2968989516 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8010830 con 0x7f05c81019d0 2024-09-19T07:33:07.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.414+0000 7f05cd200640 1 -- 172.21.15.159:0/2968989516 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c81a6ff0 con 0x7f05c81019d0 2024-09-19T07:33:07.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.415+0000 7f05cd200640 1 -- 172.21.15.159:0/2968989516 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c8101e30 con 0x7f05c81019d0 2024-09-19T07:33:07.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.419+0000 7f05c4c00640 1 -- 172.21.15.159:0/2968989516 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f05b8005020 con 0x7f05c81019d0 2024-09-19T07:33:07.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.420+0000 7f05c4c00640 1 --2- 172.21.15.159:0/2968989516 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f05ac0a3350 0x7f05ac0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:07.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.420+0000 7f05c4c00640 1 -- 172.21.15.159:0/2968989516 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f05b80d0710 con 0x7f05c81019d0 2024-09-19T07:33:07.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.420+0000 7f05bea00640 1 --2- 172.21.15.159:0/2968989516 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f05ac0a3350 0x7f05ac0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:07.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.421+0000 7f05c4c00640 1 -- 172.21.15.159:0/2968989516 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f05b809cec0 con 0x7f05c81019d0 2024-09-19T07:33:07.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.421+0000 7f05bea00640 1 --2- 172.21.15.159:0/2968989516 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f05ac0a3350 0x7f05ac0a5810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f05b0005d00 tx=0x7f05b0005c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:07.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.645+0000 7f05cd200640 1 -- 172.21.15.159:0/2968989516 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05c819b7d0 con 0x7f05c81019d0 2024-09-19T07:33:07.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.646+0000 7f05c4c00640 1 -- 172.21.15.159:0/2968989516 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11936 (secure 0 0 0) 0x7f05b809c7c0 con 0x7f05c81019d0 2024-09-19T07:33:07.647 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:33:07.647 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":25,"btime":"2024-09-19T07:31:54:895489+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3464449771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3464449771},{"type":"v1","addr":"172.21.15.159:6837","nonce":3464449771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2036264856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2036264856},{"type":"v1","addr":"172.21.15.159:6839","nonce":2036264856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6841/2874566405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6840","nonce":2874566405},{"type":"v1","addr":"172.21.15.159:6841","nonce":2874566405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:31:03.518840+0000","modified":"2024-09-19T07:31:53.954806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24542,"mds_1":24514,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/1340279549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1340279549},{"type":"v1","addr":"172.21.15.189:6835","nonce":1340279549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/937970878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":937970878},{"type":"v1","addr":"172.21.15.203:6835","nonce":937970878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1053443810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1053443810},{"type":"v1","addr":"172.21.15.203:6837","nonce":1053443810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.189:6837/1030825585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1030825585},{"type":"v1","addr":"172.21.15.189:6837","nonce":1030825585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.203:6839/3862781462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":3862781462},{"type":"v1","addr":"172.21.15.203:6839","nonce":3862781462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.189:6839/2361818663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2361818663},{"type":"v1","addr":"172.21.15.189:6839","nonce":2361818663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24506,24514,24542]},"id":1}]} 2024-09-19T07:33:07.647 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 25 2024-09-19T07:33:07.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.648+0000 7f05cd200640 1 -- 172.21.15.159:0/2968989516 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f05ac0a3350 msgr2=0x7f05ac0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:07.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.648+0000 7f05cd200640 1 --2- 172.21.15.159:0/2968989516 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f05ac0a3350 0x7f05ac0a5810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f05b0005d00 tx=0x7f05b0005c50 comp rx=0 tx=0).stop 2024-09-19T07:33:07.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.648+0000 7f05cd200640 1 -- 172.21.15.159:0/2968989516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05c81019d0 msgr2=0x7f05c819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:07.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.648+0000 7f05cd200640 1 --2- 172.21.15.159:0/2968989516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05c81019d0 0x7f05c819a470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f05b800eb90 tx=0x7f05b800c6a0 comp rx=0 tx=0).stop 2024-09-19T07:33:07.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.648+0000 7f05cd200640 1 -- 172.21.15.159:0/2968989516 shutdown_connections 2024-09-19T07:33:07.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.649+0000 7f05cd200640 1 --2- 172.21.15.159:0/2968989516 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f05ac0a3350 0x7f05ac0a5810 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:07.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.649+0000 7f05cd200640 1 --2- 172.21.15.159:0/2968989516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05c81079d0 0x7f05c819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:07.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.649+0000 7f05cd200640 1 --2- 172.21.15.159:0/2968989516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c8102370 0x7f05c819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:07.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.649+0000 7f05cd200640 1 --2- 172.21.15.159:0/2968989516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05c81019d0 0x7f05c819a470 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:07.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.649+0000 7f05cd200640 1 -- 172.21.15.159:0/2968989516 >> 172.21.15.159:0/2968989516 conn(0x7f05c80fd710 msgr2=0x7f05c80fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:07.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.649+0000 7f05cd200640 1 -- 172.21.15.159:0/2968989516 shutdown_connections 2024-09-19T07:33:07.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:07.649+0000 7f05cd200640 1 -- 172.21.15.159:0/2968989516 wait complete. 2024-09-19T07:33:08.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:08.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:08.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:07 smithi203 ceph-mon[35335]: pgmap v378: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:33:08.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:08.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:08.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:08.037 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:07 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/2968989516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:33:08.041 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:08.041 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:08.041 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:07 smithi159 ceph-mon[30472]: pgmap v378: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:33:08.041 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:08.042 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:08.042 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:08.042 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:07 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2968989516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:33:08.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:08.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:08.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:07 smithi189 ceph-mon[34851]: pgmap v378: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:33:08.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:08.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:08.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:33:08.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:07 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/2968989516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:33:08.461 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:33:08.830 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:09.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.396+0000 7f86e5800640 1 -- 172.21.15.159:0/690496558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86e01047e0 msgr2=0x7f86e0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.396+0000 7f86e5800640 1 --2- 172.21.15.159:0/690496558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86e01047e0 0x7f86e0108830 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009900 tx=0x7f86c802a410 comp rx=0 tx=0).stop 2024-09-19T07:33:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.396+0000 7f86e5800640 1 -- 172.21.15.159:0/690496558 shutdown_connections 2024-09-19T07:33:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.396+0000 7f86e5800640 1 --2- 172.21.15.159:0/690496558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86e0108f60 0x7f86e0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.396+0000 7f86e5800640 1 --2- 172.21.15.159:0/690496558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86e01047e0 0x7f86e0108830 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.396+0000 7f86e5800640 1 --2- 172.21.15.159:0/690496558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86e0103e30 0x7f86e0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.396+0000 7f86e5800640 1 -- 172.21.15.159:0/690496558 >> 172.21.15.159:0/690496558 conn(0x7f86e00ff910 msgr2=0x7f86e0101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.396+0000 7f86e5800640 1 -- 172.21.15.159:0/690496558 shutdown_connections 2024-09-19T07:33:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.397+0000 7f86e5800640 1 -- 172.21.15.159:0/690496558 wait complete. 2024-09-19T07:33:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.397+0000 7f86e5800640 1 Processor -- start 2024-09-19T07:33:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.398+0000 7f86e5800640 1 -- start start 2024-09-19T07:33:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.398+0000 7f86e5800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86e0103e30 0x7f86e019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.398+0000 7f86e5800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86e01047e0 0x7f86e019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.398+0000 7f86e5800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86e0108f60 0x7f86e019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.398+0000 7f86e5800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f86e0113fc0 con 0x7f86e0103e30 2024-09-19T07:33:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.398+0000 7f86e5800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f86e0113e40 con 0x7f86e01047e0 2024-09-19T07:33:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.398+0000 7f86e5800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f86e0114140 con 0x7f86e0108f60 2024-09-19T07:33:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.398+0000 7f86dfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86e0103e30 0x7f86e019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.398+0000 7f86d7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86e01047e0 0x7f86e019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.398+0000 7f86dfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86e0103e30 0x7f86e019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60998/0 (socket says 172.21.15.159:60998) 2024-09-19T07:33:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.398+0000 7f86dfe00640 1 -- 172.21.15.159:0/1832590694 learned_addr learned my addr 172.21.15.159:0/1832590694 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.398+0000 7f86e4e00640 1 --2- 172.21.15.159:0/1832590694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86e0108f60 0x7f86e019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.399+0000 7f86d7e00640 1 -- 172.21.15.159:0/1832590694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86e0108f60 msgr2=0x7f86e019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.399+0000 7f86d7e00640 1 --2- 172.21.15.159:0/1832590694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86e0108f60 0x7f86e019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.399+0000 7f86d7e00640 1 -- 172.21.15.159:0/1832590694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86e0103e30 msgr2=0x7f86e019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.399+0000 7f86d7e00640 1 --2- 172.21.15.159:0/1832590694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86e0103e30 0x7f86e019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.399+0000 7f86d7e00640 1 -- 172.21.15.159:0/1832590694 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e019f430 con 0x7f86e01047e0 2024-09-19T07:33:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.399+0000 7f86dfe00640 1 --2- 172.21.15.159:0/1832590694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86e0103e30 0x7f86e019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.399+0000 7f86d7e00640 1 --2- 172.21.15.159:0/1832590694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86e01047e0 0x7f86e019a9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f86c8005940 tx=0x7f86c80040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.399+0000 7f86dd600640 1 -- 172.21.15.159:0/1832590694 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c803f030 con 0x7f86e01047e0 2024-09-19T07:33:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.399+0000 7f86e5800640 1 -- 172.21.15.159:0/1832590694 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e019f6c0 con 0x7f86e01047e0 2024-09-19T07:33:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.399+0000 7f86dd600640 1 -- 172.21.15.159:0/1832590694 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86c80044c0 con 0x7f86e01047e0 2024-09-19T07:33:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.399+0000 7f86dd600640 1 -- 172.21.15.159:0/1832590694 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8035070 con 0x7f86e01047e0 2024-09-19T07:33:09.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.400+0000 7f86e5800640 1 -- 172.21.15.159:0/1832590694 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e01a7000 con 0x7f86e01047e0 2024-09-19T07:33:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.400+0000 7f86e5800640 1 -- 172.21.15.159:0/1832590694 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e0105310 con 0x7f86e01047e0 2024-09-19T07:33:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.403+0000 7f86dd600640 1 -- 172.21.15.159:0/1832590694 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f86c8042030 con 0x7f86e01047e0 2024-09-19T07:33:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.404+0000 7f86dd600640 1 --2- 172.21.15.159:0/1832590694 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f86c40a32d0 0x7f86c40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.404+0000 7f86dd600640 1 -- 172.21.15.159:0/1832590694 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f86c80f0400 con 0x7f86e01047e0 2024-09-19T07:33:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.404+0000 7f86dfe00640 1 --2- 172.21.15.159:0/1832590694 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f86c40a32d0 0x7f86c40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.405+0000 7f86dfe00640 1 --2- 172.21.15.159:0/1832590694 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f86c40a32d0 0x7f86c40a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f86d0011550 tx=0x7f86d0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.406+0000 7f86dd600640 1 -- 172.21.15.159:0/1832590694 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f86c80bc560 con 0x7f86e01047e0 2024-09-19T07:33:09.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.581+0000 7f86e5800640 1 -- 172.21.15.159:0/1832590694 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86e0104210 con 0x7f86e01047e0 2024-09-19T07:33:09.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.583+0000 7f86dd600640 1 -- 172.21.15.159:0/1832590694 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24963 (secure 0 0 0) 0x7f86e0104210 con 0x7f86e01047e0 2024-09-19T07:33:09.584 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:33:09.585 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":100,"fsid":"c9d18048-7657-11ef-bceb-c7b262605968","created":"2024-09-19T07:22:32.059836+0000","modified":"2024-09-19T07:32:31.832496+0000","last_up_change":"2024-09-19T07:30:03.328697+0000","last_in_change":"2024-09-19T07:29:38.987579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:25:55.202457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:30:58.022056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:31:01.042290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:31:10.977289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"752f5466-fe04-408a-8566-86625947fd3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6803","nonce":2738900830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6805","nonce":2738900830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6809","nonce":2738900830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6807","nonce":2738900830}]},"public_addr":"172.21.15.159:6803/2738900830","cluster_addr":"172.21.15.159:6805/2738900830","heartbeat_back_addr":"172.21.15.159:6809/2738900830","heartbeat_front_addr":"172.21.15.159:6807/2738900830","state":["exists","up"]},{"osd":1,"uuid":"7513adda-ca82-4df4-aa99-3b050b410424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6801","nonce":265696178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6803","nonce":265696178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6807","nonce":265696178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6805","nonce":265696178}]},"public_addr":"172.21.15.189:6801/265696178","cluster_addr":"172.21.15.189:6803/265696178","heartbeat_back_addr":"172.21.15.189:6807/265696178","heartbeat_front_addr":"172.21.15.189:6805/265696178","state":["exists","up"]},{"osd":2,"uuid":"aa03be79-58d8-4eb7-a051-0b46c502b12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6801","nonce":1826565288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6803","nonce":1826565288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6807","nonce":1826565288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6805","nonce":1826565288}]},"public_addr":"172.21.15.203:6801/1826565288","cluster_addr":"172.21.15.203:6803/1826565288","heartbeat_back_addr":"172.21.15.203:6807/1826565288","heartbeat_front_addr":"172.21.15.203:6805/1826565288","state":["exists","up"]},{"osd":3,"uuid":"fcd734f1-f40f-4566-82ab-c4607039f23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6811","nonce":109457380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6813","nonce":109457380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6817","nonce":109457380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6815","nonce":109457380}]},"public_addr":"172.21.15.159:6811/109457380","cluster_addr":"172.21.15.159:6813/109457380","heartbeat_back_addr":"172.21.15.159:6817/109457380","heartbeat_front_addr":"172.21.15.159:6815/109457380","state":["exists","up"]},{"osd":4,"uuid":"bc344c9e-2267-4058-bbe1-9c270327db8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6809","nonce":1036888180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6811","nonce":1036888180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6815","nonce":1036888180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6813","nonce":1036888180}]},"public_addr":"172.21.15.189:6809/1036888180","cluster_addr":"172.21.15.189:6811/1036888180","heartbeat_back_addr":"172.21.15.189:6815/1036888180","heartbeat_front_addr":"172.21.15.189:6813/1036888180","state":["exists","up"]},{"osd":5,"uuid":"86c48303-929d-42f4-bf48-cbb32649f95f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6809","nonce":4212601512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6811","nonce":4212601512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6815","nonce":4212601512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6813","nonce":4212601512}]},"public_addr":"172.21.15.203:6809/4212601512","cluster_addr":"172.21.15.203:6811/4212601512","heartbeat_back_addr":"172.21.15.203:6815/4212601512","heartbeat_front_addr":"172.21.15.203:6813/4212601512","state":["exists","up"]},{"osd":6,"uuid":"f0176ed1-c3a7-4d75-b213-fe88fab4441a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6819","nonce":3817684578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6821","nonce":3817684578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6825","nonce":3817684578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6823","nonce":3817684578}]},"public_addr":"172.21.15.159:6819/3817684578","cluster_addr":"172.21.15.159:6821/3817684578","heartbeat_back_addr":"172.21.15.159:6825/3817684578","heartbeat_front_addr":"172.21.15.159:6823/3817684578","state":["exists","up"]},{"osd":7,"uuid":"921bd59d-9415-4559-8e9f-035d336afd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6817","nonce":3936826456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6819","nonce":3936826456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6823","nonce":3936826456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6821","nonce":3936826456}]},"public_addr":"172.21.15.189:6817/3936826456","cluster_addr":"172.21.15.189:6819/3936826456","heartbeat_back_addr":"172.21.15.189:6823/3936826456","heartbeat_front_addr":"172.21.15.189:6821/3936826456","state":["exists","up"]},{"osd":8,"uuid":"ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6817","nonce":4062926645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6819","nonce":4062926645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6823","nonce":4062926645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6821","nonce":4062926645}]},"public_addr":"172.21.15.203:6817/4062926645","cluster_addr":"172.21.15.203:6819/4062926645","heartbeat_back_addr":"172.21.15.203:6823/4062926645","heartbeat_front_addr":"172.21.15.203:6821/4062926645","state":["exists","up"]},{"osd":9,"uuid":"df29a10f-1c60-41cb-9c2d-bc9f1cb174bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6827","nonce":4185848747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6829","nonce":4185848747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6833","nonce":4185848747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6831","nonce":4185848747}]},"public_addr":"172.21.15.159:6827/4185848747","cluster_addr":"172.21.15.159:6829/4185848747","heartbeat_back_addr":"172.21.15.159:6833/4185848747","heartbeat_front_addr":"172.21.15.159:6831/4185848747","state":["exists","up"]},{"osd":10,"uuid":"ccc440b4-4682-4998-bb53-95e24423cd1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6825","nonce":3018377605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6827","nonce":3018377605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6831","nonce":3018377605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6829","nonce":3018377605}]},"public_addr":"172.21.15.189:6825/3018377605","cluster_addr":"172.21.15.189:6827/3018377605","heartbeat_back_addr":"172.21.15.189:6831/3018377605","heartbeat_front_addr":"172.21.15.189:6829/3018377605","state":["exists","up"]},{"osd":11,"uuid":"e84afc5f-8051-4341-8fd0-3afb482ab4df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6825","nonce":2807503891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6827","nonce":2807503891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6831","nonce":2807503891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6829","nonce":2807503891}]},"public_addr":"172.21.15.203:6825/2807503891","cluster_addr":"172.21.15.203:6827/2807503891","heartbeat_back_addr":"172.21.15.203:6831/2807503891","heartbeat_front_addr":"172.21.15.203:6829/2807503891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:00.093677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:25.477205+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:51.345020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:21.077603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:48.822640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:13.516029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:44.382228+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:10.810700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:36.817155+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:07.728553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:34.747540+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:01.249332+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:0/2383222735":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1561371050":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6800/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:6800/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/2108730247":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/3393992615":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1366863724":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/1628344860":"2024-09-20T07:23:03.089190+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:33:09.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.586+0000 7f86e5800640 1 -- 172.21.15.159:0/1832590694 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f86c40a32d0 msgr2=0x7f86c40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:09.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.586+0000 7f86e5800640 1 --2- 172.21.15.159:0/1832590694 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f86c40a32d0 0x7f86c40a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f86d0011550 tx=0x7f86d0005f90 comp rx=0 tx=0).stop 2024-09-19T07:33:09.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.586+0000 7f86e5800640 1 -- 172.21.15.159:0/1832590694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86e01047e0 msgr2=0x7f86e019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:09.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.586+0000 7f86e5800640 1 --2- 172.21.15.159:0/1832590694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86e01047e0 0x7f86e019a9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f86c8005940 tx=0x7f86c80040c0 comp rx=0 tx=0).stop 2024-09-19T07:33:09.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.586+0000 7f86e5800640 1 -- 172.21.15.159:0/1832590694 shutdown_connections 2024-09-19T07:33:09.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.586+0000 7f86e5800640 1 --2- 172.21.15.159:0/1832590694 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f86c40a32d0 0x7f86c40a5790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:09.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.586+0000 7f86e5800640 1 --2- 172.21.15.159:0/1832590694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86e0108f60 0x7f86e019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:09.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.586+0000 7f86e5800640 1 --2- 172.21.15.159:0/1832590694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86e01047e0 0x7f86e019a9c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:09.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.586+0000 7f86e5800640 1 --2- 172.21.15.159:0/1832590694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86e0103e30 0x7f86e019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:09.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.586+0000 7f86e5800640 1 -- 172.21.15.159:0/1832590694 >> 172.21.15.159:0/1832590694 conn(0x7f86e00ff910 msgr2=0x7f86e0100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:09.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.586+0000 7f86e5800640 1 -- 172.21.15.159:0/1832590694 shutdown_connections 2024-09-19T07:33:09.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:09.586+0000 7f86e5800640 1 -- 172.21.15.159:0/1832590694 wait complete. 2024-09-19T07:33:09.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:09 smithi159 ceph-mon[30472]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:33:09.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:09 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1832590694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:33:10.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:09 smithi203 ceph-mon[35335]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:33:10.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:09 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1832590694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:33:10.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:09 smithi189 ceph-mon[34851]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:33:10.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:09 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1832590694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:33:10.381 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:33:10.755 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.371+0000 7fdf7f400640 1 -- 172.21.15.159:0/3834963078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf78101de0 msgr2=0x7fdf781021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.371+0000 7fdf7f400640 1 --2- 172.21.15.159:0/3834963078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf78101de0 0x7fdf781021c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdf68009900 tx=0x7fdf6802a3f0 comp rx=0 tx=0).stop 2024-09-19T07:33:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.372+0000 7fdf7f400640 1 -- 172.21.15.159:0/3834963078 shutdown_connections 2024-09-19T07:33:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.372+0000 7fdf7f400640 1 --2- 172.21.15.159:0/3834963078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf78106f10 0x7fdf7810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:11.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.372+0000 7fdf7f400640 1 --2- 172.21.15.159:0/3834963078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf78102790 0x7fdf781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:11.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.372+0000 7fdf7f400640 1 --2- 172.21.15.159:0/3834963078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf78101de0 0x7fdf781021c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:11.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.372+0000 7fdf7f400640 1 -- 172.21.15.159:0/3834963078 >> 172.21.15.159:0/3834963078 conn(0x7fdf780fd8c0 msgr2=0x7fdf780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:11.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.372+0000 7fdf7f400640 1 -- 172.21.15.159:0/3834963078 shutdown_connections 2024-09-19T07:33:11.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.372+0000 7fdf7f400640 1 -- 172.21.15.159:0/3834963078 wait complete. 2024-09-19T07:33:11.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.373+0000 7fdf7f400640 1 Processor -- start 2024-09-19T07:33:11.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.373+0000 7fdf7f400640 1 -- start start 2024-09-19T07:33:11.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.373+0000 7fdf7f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf78101de0 0x7fdf780744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:11.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.373+0000 7fdf7f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf78102790 0x7fdf78074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:11.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.373+0000 7fdf7f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf78106f10 0x7fdf78074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.373+0000 7fdf7f400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdf78111fc0 con 0x7fdf78102790 2024-09-19T07:33:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.373+0000 7fdf7f400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdf78111e40 con 0x7fdf78106f10 2024-09-19T07:33:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.373+0000 7fdf7f400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdf78112140 con 0x7fdf78101de0 2024-09-19T07:33:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.373+0000 7fdf6fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf78102790 0x7fdf78074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.374+0000 7fdf6fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf78102790 0x7fdf78074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:32788/0 (socket says 172.21.15.159:32788) 2024-09-19T07:33:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.374+0000 7fdf6fe00640 1 -- 172.21.15.159:0/1866182815 learned_addr learned my addr 172.21.15.159:0/1866182815 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.374+0000 7fdf7ca00640 1 --2- 172.21.15.159:0/1866182815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf78101de0 0x7fdf780744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.374+0000 7fdf7d400640 1 --2- 172.21.15.159:0/1866182815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf78106f10 0x7fdf78074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.374+0000 7fdf6fe00640 1 -- 172.21.15.159:0/1866182815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf78101de0 msgr2=0x7fdf780744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.374+0000 7fdf6fe00640 1 --2- 172.21.15.159:0/1866182815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf78101de0 0x7fdf780744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.374+0000 7fdf6fe00640 1 -- 172.21.15.159:0/1866182815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf78106f10 msgr2=0x7fdf78074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.374+0000 7fdf6fe00640 1 --2- 172.21.15.159:0/1866182815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf78106f10 0x7fdf78074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.374+0000 7fdf6fe00640 1 -- 172.21.15.159:0/1866182815 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf781a68b0 con 0x7fdf78102790 2024-09-19T07:33:11.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.374+0000 7fdf6fe00640 1 --2- 172.21.15.159:0/1866182815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf78102790 0x7fdf78074a30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000d950 tx=0x7fdf6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:11.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.375+0000 7fdf76000640 1 -- 172.21.15.159:0/1866182815 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf60017030 con 0x7fdf78102790 2024-09-19T07:33:11.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.375+0000 7fdf76000640 1 -- 172.21.15.159:0/1866182815 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf60004020 con 0x7fdf78102790 2024-09-19T07:33:11.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.375+0000 7fdf7f400640 1 -- 172.21.15.159:0/1866182815 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf781a6ba0 con 0x7fdf78102790 2024-09-19T07:33:11.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.375+0000 7fdf76000640 1 -- 172.21.15.159:0/1866182815 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf600047a0 con 0x7fdf78102790 2024-09-19T07:33:11.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.375+0000 7fdf7f400640 1 -- 172.21.15.159:0/1866182815 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf781a7130 con 0x7fdf78102790 2024-09-19T07:33:11.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.376+0000 7fdf7f400640 1 -- 172.21.15.159:0/1866182815 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf781021c0 con 0x7fdf78102790 2024-09-19T07:33:11.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.378+0000 7fdf76000640 1 -- 172.21.15.159:0/1866182815 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fdf600041d0 con 0x7fdf78102790 2024-09-19T07:33:11.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.379+0000 7fdf76000640 1 --2- 172.21.15.159:0/1866182815 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdf5c0a3350 0x7fdf5c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:11.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.379+0000 7fdf76000640 1 -- 172.21.15.159:0/1866182815 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fdf600cf760 con 0x7fdf78102790 2024-09-19T07:33:11.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.380+0000 7fdf7ca00640 1 --2- 172.21.15.159:0/1866182815 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdf5c0a3350 0x7fdf5c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:11.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.380+0000 7fdf7ca00640 1 --2- 172.21.15.159:0/1866182815 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdf5c0a3350 0x7fdf5c0a5810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdf68000c00 tx=0x7fdf68005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:11.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.384+0000 7fdf76000640 1 -- 172.21.15.159:0/1866182815 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdf6009bf10 con 0x7fdf78102790 2024-09-19T07:33:11.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.592+0000 7fdf7f400640 1 -- 172.21.15.159:0/1866182815 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf78109b60 con 0x7fdf78102790 2024-09-19T07:33:11.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.593+0000 7fdf76000640 1 -- 172.21.15.159:0/1866182815 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11936 (secure 0 0 0) 0x7fdf6009b810 con 0x7fdf78102790 2024-09-19T07:33:11.594 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:33:11.594 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":25,"btime":"2024-09-19T07:31:54:895489+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3464449771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3464449771},{"type":"v1","addr":"172.21.15.159:6837","nonce":3464449771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2036264856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2036264856},{"type":"v1","addr":"172.21.15.159:6839","nonce":2036264856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6841/2874566405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6840","nonce":2874566405},{"type":"v1","addr":"172.21.15.159:6841","nonce":2874566405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:31:03.518840+0000","modified":"2024-09-19T07:31:53.954806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24542,"mds_1":24514,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/1340279549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1340279549},{"type":"v1","addr":"172.21.15.189:6835","nonce":1340279549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/937970878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":937970878},{"type":"v1","addr":"172.21.15.203:6835","nonce":937970878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1053443810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1053443810},{"type":"v1","addr":"172.21.15.203:6837","nonce":1053443810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.189:6837/1030825585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1030825585},{"type":"v1","addr":"172.21.15.189:6837","nonce":1030825585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.203:6839/3862781462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":3862781462},{"type":"v1","addr":"172.21.15.203:6839","nonce":3862781462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.189:6839/2361818663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2361818663},{"type":"v1","addr":"172.21.15.189:6839","nonce":2361818663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24506,24514,24542]},"id":1}]} 2024-09-19T07:33:11.594 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 25 2024-09-19T07:33:11.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.595+0000 7fdf7f400640 1 -- 172.21.15.159:0/1866182815 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdf5c0a3350 msgr2=0x7fdf5c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:11.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.595+0000 7fdf7f400640 1 --2- 172.21.15.159:0/1866182815 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdf5c0a3350 0x7fdf5c0a5810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdf68000c00 tx=0x7fdf68005b50 comp rx=0 tx=0).stop 2024-09-19T07:33:11.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.595+0000 7fdf7f400640 1 -- 172.21.15.159:0/1866182815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf78102790 msgr2=0x7fdf78074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:11.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.595+0000 7fdf7f400640 1 --2- 172.21.15.159:0/1866182815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf78102790 0x7fdf78074a30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000d950 tx=0x7fdf6000de10 comp rx=0 tx=0).stop 2024-09-19T07:33:11.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.595+0000 7fdf7f400640 1 -- 172.21.15.159:0/1866182815 shutdown_connections 2024-09-19T07:33:11.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.595+0000 7fdf7f400640 1 --2- 172.21.15.159:0/1866182815 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdf5c0a3350 0x7fdf5c0a5810 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:11.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.595+0000 7fdf7f400640 1 --2- 172.21.15.159:0/1866182815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf78106f10 0x7fdf78074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:11.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.595+0000 7fdf7f400640 1 --2- 172.21.15.159:0/1866182815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf78102790 0x7fdf78074a30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:11.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.595+0000 7fdf7f400640 1 --2- 172.21.15.159:0/1866182815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf78101de0 0x7fdf780744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:11.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.595+0000 7fdf7f400640 1 -- 172.21.15.159:0/1866182815 >> 172.21.15.159:0/1866182815 conn(0x7fdf780fd8c0 msgr2=0x7fdf780febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:11.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.595+0000 7fdf7f400640 1 -- 172.21.15.159:0/1866182815 shutdown_connections 2024-09-19T07:33:11.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:11.595+0000 7fdf7f400640 1 -- 172.21.15.159:0/1866182815 wait complete. 2024-09-19T07:33:11.886 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:11 smithi159 ceph-mon[30472]: pgmap v380: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:11.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:11 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1866182815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:33:12.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:11 smithi203 ceph-mon[35335]: pgmap v380: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:12.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:11 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1866182815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:33:12.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:11 smithi189 ceph-mon[34851]: pgmap v380: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:12.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:11 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1866182815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:33:12.479 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T07:31:03.518840+0000', 'modified': '2024-09-19T07:31:53.954806+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24542, 'mds_1': 24514, 'mds_2': 24506}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6835/1340279549', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6834', 'nonce': 1340279549}, {'type': 'v1', 'addr': '172.21.15.189:6835', 'nonce': 1340279549}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6835/937970878', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 937970878}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 937970878}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6837/1053443810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 1053443810}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 1053443810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.189:6837/1030825585', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 1030825585}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 1030825585}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.203:6839/3862781462', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6838', 'nonce': 3862781462}, {'type': 'v1', 'addr': '172.21.15.203:6839', 'nonce': 3862781462}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24542': {'gid': 24542, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.189:6839/2361818663', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 2361818663}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 2361818663}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24542, 'qdb_cluster': [24506, 24514, 24542]} 2024-09-19T07:33:12.479 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T07:33:12.479 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T07:33:12.852 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:13.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.406+0000 7fcb27e00640 1 -- 172.21.15.159:0/3430833105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb2010e3c0 msgr2=0x7fcb20110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:13.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.406+0000 7fcb27e00640 1 --2- 172.21.15.159:0/3430833105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb2010e3c0 0x7fcb20110810 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009ca0 tx=0x7fcb1802a950 comp rx=0 tx=0).stop 2024-09-19T07:33:13.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.406+0000 7fcb27e00640 1 -- 172.21.15.159:0/3430833105 shutdown_connections 2024-09-19T07:33:13.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.406+0000 7fcb27e00640 1 --2- 172.21.15.159:0/3430833105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb2010e3c0 0x7fcb20110810 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.406+0000 7fcb27e00640 1 --2- 172.21.15.159:0/3430833105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb20074f40 0x7fcb2010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.406+0000 7fcb27e00640 1 --2- 172.21.15.159:0/3430833105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb20074620 0x7fcb20074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.406+0000 7fcb27e00640 1 -- 172.21.15.159:0/3430833105 >> 172.21.15.159:0/3430833105 conn(0x7fcb200fd8c0 msgr2=0x7fcb200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:13.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.407+0000 7fcb27e00640 1 -- 172.21.15.159:0/3430833105 shutdown_connections 2024-09-19T07:33:13.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.407+0000 7fcb27e00640 1 -- 172.21.15.159:0/3430833105 wait complete. 2024-09-19T07:33:13.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.407+0000 7fcb27e00640 1 Processor -- start 2024-09-19T07:33:13.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.408+0000 7fcb27e00640 1 -- start start 2024-09-19T07:33:13.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.408+0000 7fcb27e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb20074620 0x7fcb2019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.408+0000 7fcb27e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb20074f40 0x7fcb2019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.408+0000 7fcb27e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb2010e3c0 0x7fcb2019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.408+0000 7fcb27e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcb20114160 con 0x7fcb20074f40 2024-09-19T07:33:13.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.408+0000 7fcb27e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcb20113fe0 con 0x7fcb2010e3c0 2024-09-19T07:33:13.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.408+0000 7fcb27e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcb201142e0 con 0x7fcb20074620 2024-09-19T07:33:13.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.408+0000 7fcb24a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb20074f40 0x7fcb2019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.408+0000 7fcb25400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb20074620 0x7fcb2019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.408+0000 7fcb24a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb20074f40 0x7fcb2019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:57032/0 (socket says 172.21.15.159:57032) 2024-09-19T07:33:13.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.408+0000 7fcb24a00640 1 -- 172.21.15.159:0/3460081727 learned_addr learned my addr 172.21.15.159:0/3460081727 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:13.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.408+0000 7fcb25e00640 1 --2- 172.21.15.159:0/3460081727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb2010e3c0 0x7fcb2019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.409+0000 7fcb25400640 1 -- 172.21.15.159:0/3460081727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb2010e3c0 msgr2=0x7fcb2019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:13.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.409+0000 7fcb25400640 1 --2- 172.21.15.159:0/3460081727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb2010e3c0 0x7fcb2019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.409+0000 7fcb25400640 1 -- 172.21.15.159:0/3460081727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb20074f40 msgr2=0x7fcb2019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:13.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.409+0000 7fcb25400640 1 --2- 172.21.15.159:0/3460081727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb20074f40 0x7fcb2019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.409+0000 7fcb25400640 1 -- 172.21.15.159:0/3460081727 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2019f610 con 0x7fcb20074620 2024-09-19T07:33:13.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.409+0000 7fcb25400640 1 --2- 172.21.15.159:0/3460081727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb20074620 0x7fcb2019a660 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400e9f0 tx=0x7fcb1400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:13.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.409+0000 7fcb1e000640 1 -- 172.21.15.159:0/3460081727 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1400ccc0 con 0x7fcb20074620 2024-09-19T07:33:13.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.409+0000 7fcb1e000640 1 -- 172.21.15.159:0/3460081727 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb14005020 con 0x7fcb20074620 2024-09-19T07:33:13.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.410+0000 7fcb27e00640 1 -- 172.21.15.159:0/3460081727 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2019f900 con 0x7fcb20074620 2024-09-19T07:33:13.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.410+0000 7fcb1e000640 1 -- 172.21.15.159:0/3460081727 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb140049e0 con 0x7fcb20074620 2024-09-19T07:33:13.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.410+0000 7fcb27e00640 1 -- 172.21.15.159:0/3460081727 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb201a71e0 con 0x7fcb20074620 2024-09-19T07:33:13.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.413+0000 7fcb2a818740 1 Processor -- start 2024-09-19T07:33:13.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.413+0000 7fcb2a818740 1 -- start start 2024-09-19T07:33:13.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.413+0000 7fcb1e000640 1 -- 172.21.15.159:0/3460081727 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fcb14005560 con 0x7fcb20074620 2024-09-19T07:33:13.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.413+0000 7fcb2a818740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eee8330 0x555d8eee8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.413+0000 7fcb2a818740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eee8c50 0x555d8eef2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.413+0000 7fcb2a818740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eef3510 0x555d8eef5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.413+0000 7fcb2a818740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d8ee4e0b0 con 0x555d8eef3510 2024-09-19T07:33:13.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.413+0000 7fcb2a818740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555d8edf5ab0 con 0x555d8eee8c50 2024-09-19T07:33:13.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.413+0000 7fcb2a818740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555d8ee4a070 con 0x555d8eee8330 2024-09-19T07:33:13.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.413+0000 7fcb25400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eee8c50 0x555d8eef2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.413+0000 7fcb25e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eee8330 0x555d8eee8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.413+0000 7fcb24a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eef3510 0x555d8eef5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.413+0000 7fcb25400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eee8c50 0x555d8eef2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:38958/0 (socket says 172.21.15.159:38958) 2024-09-19T07:33:13.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.413+0000 7fcb25400640 1 -- 172.21.15.159:0/2032913649 learned_addr learned my addr 172.21.15.159:0/2032913649 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:13.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.414+0000 7fcb25e00640 1 -- 172.21.15.159:0/2032913649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eee8c50 msgr2=0x555d8eef2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:13.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.414+0000 7fcb25e00640 1 --2- 172.21.15.159:0/2032913649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eee8c50 0x555d8eef2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.414+0000 7fcb25e00640 1 -- 172.21.15.159:0/2032913649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eef3510 msgr2=0x555d8eef5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:13.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.414+0000 7fcb25e00640 1 --2- 172.21.15.159:0/2032913649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eef3510 0x555d8eef5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.414+0000 7fcb25e00640 1 -- 172.21.15.159:0/2032913649 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d8ee024a0 con 0x555d8eee8330 2024-09-19T07:33:13.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.414+0000 7fcb25e00640 1 --2- 172.21.15.159:0/2032913649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eee8330 0x555d8eee8710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcb18005cc0 tx=0x7fcb180093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3c0847ad97476 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:13.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.415+0000 7fcb12600640 1 -- 172.21.15.159:0/2032913649 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb18048070 con 0x555d8eee8330 2024-09-19T07:33:13.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.415+0000 7fcb12600640 1 -- 172.21.15.159:0/2032913649 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb1802f940 con 0x555d8eee8330 2024-09-19T07:33:13.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.415+0000 7fcb12600640 1 -- 172.21.15.159:0/2032913649 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb18034040 con 0x555d8eee8330 2024-09-19T07:33:13.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.415+0000 7fcb1e000640 1 --2- 172.21.15.159:0/3460081727 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcb080a3350 0x7fcb080a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.415+0000 7fcb2a818740 1 -- 172.21.15.159:0/2032913649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eee8330 msgr2=0x555d8eee8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:13.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.415+0000 7fcb2a818740 1 --2- 172.21.15.159:0/2032913649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eee8330 0x555d8eee8710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcb18005cc0 tx=0x7fcb180093e0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.415+0000 7fcb1e000640 1 -- 172.21.15.159:0/3460081727 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fcb140cfd80 con 0x7fcb20074620 2024-09-19T07:33:13.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.415+0000 7fcb2a818740 1 -- 172.21.15.159:0/2032913649 shutdown_connections 2024-09-19T07:33:13.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.415+0000 7fcb2a818740 1 --2- 172.21.15.159:0/2032913649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eef3510 0x555d8eef5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.415+0000 7fcb2a818740 1 --2- 172.21.15.159:0/2032913649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eee8c50 0x555d8eef2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.415+0000 7fcb2a818740 1 --2- 172.21.15.159:0/2032913649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eee8330 0x555d8eee8710 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.415+0000 7fcb2a818740 1 -- 172.21.15.159:0/2032913649 >> 172.21.15.159:0/2032913649 conn(0x555d8eee48c0 msgr2=0x555d8eee4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:13.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.415+0000 7fcb25400640 1 --2- 172.21.15.159:0/3460081727 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcb080a3350 0x7fcb080a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.416+0000 7fcb2a818740 1 -- 172.21.15.159:0/2032913649 shutdown_connections 2024-09-19T07:33:13.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.416+0000 7fcb2a818740 1 -- 172.21.15.159:0/2032913649 wait complete. 2024-09-19T07:33:13.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.416+0000 7fcb25400640 1 --2- 172.21.15.159:0/3460081727 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcb080a3350 0x7fcb080a5810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400e9c0 tx=0x7fcb140d3830 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:13.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.416+0000 7fcb2a818740 1 -- start start 2024-09-19T07:33:13.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.416+0000 7fcb2a818740 1 Processor -- start 2024-09-19T07:33:13.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.417+0000 7fcb2a818740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8330 0x555d8eef0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.417+0000 7fcb2a818740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eee8c50 0x555d8eef0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.417+0000 7fcb2a818740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eef3510 0x555d8eef1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.417+0000 7fcb2a818740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d8eef6060 con 0x555d8eee8330 2024-09-19T07:33:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.417+0000 7fcb2a818740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555d8edd6920 con 0x555d8eee8c50 2024-09-19T07:33:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.417+0000 7fcb2a818740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555d8eef61e0 con 0x555d8eef3510 2024-09-19T07:33:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.417+0000 7fcb2a818740 10 client.0 _ll_register_callbacks cb 0x555d8edfa8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:33:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.417+0000 7fcb25e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8330 0x555d8eef0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.417+0000 7fcb25e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8330 0x555d8eef0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:57042/0 (socket says 172.21.15.159:57042) 2024-09-19T07:33:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.417+0000 7fcb25e00640 1 -- 172.21.15.159:0/983737430 learned_addr learned my addr 172.21.15.159:0/983737430 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.417+0000 7fcb24a00640 1 --2- 172.21.15.159:0/983737430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eee8c50 0x555d8eef0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.417+0000 7fcb25e00640 1 --2- 172.21.15.159:0/983737430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eef3510 0x555d8eef1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.418+0000 7fcb25e00640 1 -- 172.21.15.159:0/983737430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eef3510 msgr2=0x555d8eef1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:13.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.418+0000 7fcb25e00640 1 --2- 172.21.15.159:0/983737430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eef3510 0x555d8eef1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.418+0000 7fcb25e00640 1 -- 172.21.15.159:0/983737430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eee8c50 msgr2=0x555d8eef0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:13.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.418+0000 7fcb25e00640 1 --2- 172.21.15.159:0/983737430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eee8c50 0x555d8eef0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.418+0000 7fcb25e00640 1 -- 172.21.15.159:0/983737430 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d8ebbf2b0 con 0x555d8eee8330 2024-09-19T07:33:13.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.418+0000 7fcb24a00640 1 --2- 172.21.15.159:0/983737430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eee8c50 0x555d8eef0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:13.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.418+0000 7fcb25e00640 1 --2- 172.21.15.159:0/983737430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8330 0x555d8eef0680 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcb1802ff70 tx=0x7fcb18003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc68729ce8da46d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:13.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.418+0000 7fcb06a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-19T07:33:13.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.418+0000 7fcb06a00640 1 -- 172.21.15.159:0/983737430 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb18034050 con 0x555d8eee8330 2024-09-19T07:33:13.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.418+0000 7fcb06a00640 1 -- 172.21.15.159:0/983737430 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fcb18030650 con 0x555d8eee8330 2024-09-19T07:33:13.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.418+0000 7fcb2a818740 1 -- 172.21.15.159:0/983737430 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb1802f940 con 0x555d8eee8330 2024-09-19T07:33:13.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.419+0000 7fcb06a00640 1 -- 172.21.15.159:0/983737430 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcb18059050 con 0x555d8eee8330 2024-09-19T07:33:13.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.419+0000 7fcb2a818740 10 client.14943 fetch_fsmap learned FSMap version 25 2024-09-19T07:33:13.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.419+0000 7fcb2a818740 1 -- 172.21.15.159:0/983737430 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555d8ef01e90 con 0x555d8eee8330 2024-09-19T07:33:13.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.419+0000 7fcb06a00640 1 -- 172.21.15.159:0/983737430 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb1802f940 con 0x555d8eee8330 2024-09-19T07:33:13.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.419+0000 7fcb06a00640 1 -- 172.21.15.159:0/983737430 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcb1804e020 con 0x555d8eee8330 2024-09-19T07:33:13.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.419+0000 7fcb2a818740 10 client.14943 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T07:33:13.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.419+0000 7fcb2a818740 10 client.14943 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T07:33:13.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.419+0000 7fcb2a818740 20 client.14943 populate_metadata read hostname 'smithi159' 2024-09-19T07:33:13.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.419+0000 7fcb2a818740 1 --2- 172.21.15.159:0/983737430 >> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] conn(0x555d8ef031e0 0x555d8ef05390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.419+0000 7fcb2a818740 4 client.14943 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:33:13.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.419+0000 7fcb2a818740 1 -- 172.21.15.159:0/983737430 --> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d8ee5b620 con 0x555d8ef031e0 2024-09-19T07:33:13.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.420+0000 7fcb24a00640 1 --2- 172.21.15.159:0/983737430 >> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] conn(0x555d8ef031e0 0x555d8ef05390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.420+0000 7fcb24a00640 1 --2- 172.21.15.159:0/983737430 >> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] conn(0x555d8ef031e0 0x555d8ef05390 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=91d24d9e4484ec70 server_cookie=437b4aed98fc7cf8 in_seq=0 out_seq=0 2024-09-19T07:33:13.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.420+0000 7fcb06a00640 10 client.14943 ms_handle_connect on v2:172.21.15.189:6836/1030825585 2024-09-19T07:33:13.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.421+0000 7fcb06a00640 1 -- 172.21.15.159:0/983737430 <== mds.2 v2:172.21.15.189:6836/1030825585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d8ee5b620 con 0x555d8ef031e0 2024-09-19T07:33:13.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.421+0000 7fcb06a00640 10 client.14943 handle_command_reply: tid=0 2024-09-19T07:33:13.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.421+0000 7fcb2a818740 1 client.14943 shutdown 2024-09-19T07:33:13.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.421+0000 7fcb2a818740 1 -- 172.21.15.159:0/983737430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8330 msgr2=0x555d8eef0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:13.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.421+0000 7fcb2a818740 1 --2- 172.21.15.159:0/983737430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8330 0x555d8eef0680 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcb1802ff70 tx=0x7fcb18003ca0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.421+0000 7fcb2a818740 1 -- 172.21.15.159:0/983737430 shutdown_connections 2024-09-19T07:33:13.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.421+0000 7fcb2a818740 1 --2- 172.21.15.159:0/983737430 >> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] conn(0x555d8ef031e0 0x555d8ef05390 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.421+0000 7fcb2a818740 1 --2- 172.21.15.159:0/983737430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eef3510 0x555d8eef1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.421+0000 7fcb2a818740 1 --2- 172.21.15.159:0/983737430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eee8c50 0x555d8eef0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.421+0000 7fcb2a818740 1 --2- 172.21.15.159:0/983737430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8330 0x555d8eef0680 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.421+0000 7fcb06a00640 0 client.14943 ms_handle_reset on v2:172.21.15.189:6836/1030825585 2024-09-19T07:33:13.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.421+0000 7fcb2a818740 1 -- 172.21.15.159:0/983737430 >> 172.21.15.159:0/983737430 conn(0x555d8eee48c0 msgr2=0x555d8eeea5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:13.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.422+0000 7fcb2a818740 1 -- 172.21.15.159:0/983737430 shutdown_connections 2024-09-19T07:33:13.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.422+0000 7fcb2a818740 1 -- 172.21.15.159:0/983737430 wait complete. 2024-09-19T07:33:13.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.422+0000 7fcb2a818740 20 client.14943 trim_cache size 0 max 16384 2024-09-19T07:33:13.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.440+0000 7fcb2a818740 1 Processor -- start 2024-09-19T07:33:13.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.440+0000 7fcb2a818740 1 -- start start 2024-09-19T07:33:13.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.440+0000 7fcb2a818740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eee8330 0x555d8eee4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.440+0000 7fcb2a818740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8c50 0x555d8eeeff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.440+0000 7fcb2a818740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eef3510 0x555d8eeed900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.441+0000 7fcb2a818740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d8ef036d0 con 0x555d8eee8c50 2024-09-19T07:33:13.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.441+0000 7fcb2a818740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555d8ef03da0 con 0x555d8eef3510 2024-09-19T07:33:13.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.441+0000 7fcb2a818740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555d8ef04470 con 0x555d8eee8330 2024-09-19T07:33:13.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.441+0000 7fcb24a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8c50 0x555d8eeeff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.441+0000 7fcb24a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8c50 0x555d8eeeff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:57044/0 (socket says 172.21.15.159:57044) 2024-09-19T07:33:13.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.441+0000 7fcb24a00640 1 -- 172.21.15.159:0/4282239410 learned_addr learned my addr 172.21.15.159:0/4282239410 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:13.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.441+0000 7fcb25e00640 1 --2- 172.21.15.159:0/4282239410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eee8330 0x555d8eee4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.441+0000 7fcb25e00640 1 --2- 172.21.15.159:0/4282239410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eef3510 0x555d8eeed900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.441+0000 7fcb24a00640 1 -- 172.21.15.159:0/4282239410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eee8330 msgr2=0x555d8eee4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:13.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.441+0000 7fcb24a00640 1 --2- 172.21.15.159:0/4282239410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eee8330 0x555d8eee4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.441+0000 7fcb24a00640 1 -- 172.21.15.159:0/4282239410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eef3510 msgr2=0x555d8eeed900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:13.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.441+0000 7fcb24a00640 1 --2- 172.21.15.159:0/4282239410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eef3510 0x555d8eeed900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.441+0000 7fcb24a00640 1 -- 172.21.15.159:0/4282239410 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d8eee5a50 con 0x555d8eee8c50 2024-09-19T07:33:13.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.442+0000 7fcb24a00640 1 --2- 172.21.15.159:0/4282239410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8c50 0x555d8eeeff40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009620 tx=0x7fcb0c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c226aa1de856ffe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:13.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.442+0000 7fcb11c00640 1 -- 172.21.15.159:0/4282239410 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c022070 con 0x555d8eee8c50 2024-09-19T07:33:13.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.442+0000 7fcb11c00640 1 -- 172.21.15.159:0/4282239410 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb0c0047a0 con 0x555d8eee8c50 2024-09-19T07:33:13.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.442+0000 7fcb11c00640 1 -- 172.21.15.159:0/4282239410 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c010830 con 0x555d8eee8c50 2024-09-19T07:33:13.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.443+0000 7fcb2a818740 1 -- 172.21.15.159:0/4282239410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8c50 msgr2=0x555d8eeeff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:13.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.443+0000 7fcb2a818740 1 --2- 172.21.15.159:0/4282239410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8c50 0x555d8eeeff40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009620 tx=0x7fcb0c005f90 comp rx=0 tx=0).stop 2024-09-19T07:33:13.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.443+0000 7fcb2a818740 1 -- 172.21.15.159:0/4282239410 shutdown_connections 2024-09-19T07:33:13.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.443+0000 7fcb2a818740 1 --2- 172.21.15.159:0/4282239410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eef3510 0x555d8eeed900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.443+0000 7fcb2a818740 1 --2- 172.21.15.159:0/4282239410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8c50 0x555d8eeeff40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.443+0000 7fcb2a818740 1 --2- 172.21.15.159:0/4282239410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eee8330 0x555d8eee4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.443+0000 7fcb2a818740 1 -- 172.21.15.159:0/4282239410 >> 172.21.15.159:0/4282239410 conn(0x555d8eee48c0 msgr2=0x555d8eeeef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:13.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.443+0000 7fcb2a818740 1 -- 172.21.15.159:0/4282239410 shutdown_connections 2024-09-19T07:33:13.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.443+0000 7fcb2a818740 1 -- 172.21.15.159:0/4282239410 wait complete. 2024-09-19T07:33:13.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.443+0000 7fcb2a818740 1 -- start start 2024-09-19T07:33:13.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.443+0000 7fcb2a818740 1 Processor -- start 2024-09-19T07:33:13.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.444+0000 7fcb2a818740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8330 0x555d8eeed0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.444+0000 7fcb2a818740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eee8c50 0x555d8eeed600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.444+0000 7fcb2a818740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eef3510 0x555d8ef0c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.444+0000 7fcb2a818740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d8ef05210 con 0x555d8eee8330 2024-09-19T07:33:13.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.444+0000 7fcb2a818740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555d8ef04b40 con 0x555d8eee8c50 2024-09-19T07:33:13.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.444+0000 7fcb2a818740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555d8ef058e0 con 0x555d8eef3510 2024-09-19T07:33:13.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.444+0000 7fcb2a818740 10 client.0 _ll_register_callbacks cb 0x7fcb18009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:33:13.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.444+0000 7fcb25e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8330 0x555d8eeed0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.444+0000 7fcb25e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8330 0x555d8eeed0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:57056/0 (socket says 172.21.15.159:57056) 2024-09-19T07:33:13.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.444+0000 7fcb25e00640 1 -- 172.21.15.159:0/2285945706 learned_addr learned my addr 172.21.15.159:0/2285945706 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:13.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.444+0000 7fcb24a00640 1 --2- 172.21.15.159:0/2285945706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eee8c50 0x555d8eeed600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.444+0000 7fcb25e00640 1 --2- 172.21.15.159:0/2285945706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eef3510 0x555d8ef0c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.445+0000 7fcb25e00640 1 -- 172.21.15.159:0/2285945706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eef3510 msgr2=0x555d8ef0c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:13.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.445+0000 7fcb25e00640 1 --2- 172.21.15.159:0/2285945706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eef3510 0x555d8ef0c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.445+0000 7fcb25e00640 1 -- 172.21.15.159:0/2285945706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eee8c50 msgr2=0x555d8eeed600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:13.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.445+0000 7fcb25e00640 1 --2- 172.21.15.159:0/2285945706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eee8c50 0x555d8eeed600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.445+0000 7fcb25e00640 1 -- 172.21.15.159:0/2285945706 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d8ef0cc10 con 0x555d8eee8330 2024-09-19T07:33:13.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.445+0000 7fcb25e00640 1 --2- 172.21.15.159:0/2285945706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8330 0x555d8eeed0c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009a60 tx=0x7fcb18003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44352eeaee64a77 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:13.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.445+0000 7fcb06a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-19T07:33:13.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.445+0000 7fcb06a00640 1 -- 172.21.15.159:0/2285945706 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb18034040 con 0x555d8eee8330 2024-09-19T07:33:13.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.445+0000 7fcb06a00640 1 -- 172.21.15.159:0/2285945706 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fcb18050910 con 0x555d8eee8330 2024-09-19T07:33:13.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.445+0000 7fcb2a818740 1 -- 172.21.15.159:0/2285945706 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb0c0047a0 con 0x555d8eee8330 2024-09-19T07:33:13.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.445+0000 7fcb06a00640 1 -- 172.21.15.159:0/2285945706 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcb1806e050 con 0x555d8eee8330 2024-09-19T07:33:13.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.445+0000 7fcb2a818740 10 client.14949 fetch_fsmap learned FSMap version 25 2024-09-19T07:33:13.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.445+0000 7fcb2a818740 1 -- 172.21.15.159:0/2285945706 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555d8ef0deb0 con 0x555d8eee8330 2024-09-19T07:33:13.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.446+0000 7fcb06a00640 1 -- 172.21.15.159:0/2285945706 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb0c0047a0 con 0x555d8eee8330 2024-09-19T07:33:13.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.446+0000 7fcb06a00640 1 -- 172.21.15.159:0/2285945706 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcb1804e070 con 0x555d8eee8330 2024-09-19T07:33:13.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.446+0000 7fcb2a818740 10 client.14949 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T07:33:13.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.446+0000 7fcb2a818740 10 client.14949 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T07:33:13.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.446+0000 7fcb2a818740 20 client.14949 populate_metadata read hostname 'smithi159' 2024-09-19T07:33:13.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.446+0000 7fcb2a818740 1 --2- 172.21.15.159:0/2285945706 >> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] conn(0x555d8ef7b350 0x555d8ef7a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:13.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.446+0000 7fcb2a818740 4 client.14949 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:33:13.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.446+0000 7fcb2a818740 1 -- 172.21.15.159:0/2285945706 --> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] -- command(tid 0: {"prefix": "status"}) -- 0x555d8ee55b30 con 0x555d8ef7b350 2024-09-19T07:33:13.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.446+0000 7fcb24a00640 1 --2- 172.21.15.159:0/2285945706 >> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] conn(0x555d8ef7b350 0x555d8ef7a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:13.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.447+0000 7fcb24a00640 1 --2- 172.21.15.159:0/2285945706 >> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] conn(0x555d8ef7b350 0x555d8ef7a060 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f3bb745960d3485 server_cookie=9e0d3c97f5d7b99a in_seq=0 out_seq=0 2024-09-19T07:33:13.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.447+0000 7fcb06a00640 10 client.14949 ms_handle_connect on v2:172.21.15.189:6836/1030825585 2024-09-19T07:33:13.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.447+0000 7fcb06a00640 1 -- 172.21.15.159:0/2285945706 <== mds.2 v2:172.21.15.189:6836/1030825585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x555d8ee55b30 con 0x555d8ef7b350 2024-09-19T07:33:13.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.447+0000 7fcb06a00640 10 client.14949 handle_command_reply: tid=0 2024-09-19T07:33:13.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.447+0000 7fcb2a818740 1 client.14949 shutdown 2024-09-19T07:33:13.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.447+0000 7fcb2a818740 1 -- 172.21.15.159:0/2285945706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8330 msgr2=0x555d8eeed0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:13.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.447+0000 7fcb2a818740 1 --2- 172.21.15.159:0/2285945706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8330 0x555d8eeed0c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009a60 tx=0x7fcb18003ca0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.447+0000 7fcb2a818740 1 -- 172.21.15.159:0/2285945706 shutdown_connections 2024-09-19T07:33:13.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.447+0000 7fcb2a818740 1 --2- 172.21.15.159:0/2285945706 >> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] conn(0x555d8ef7b350 0x555d8ef7a060 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.447+0000 7fcb2a818740 1 --2- 172.21.15.159:0/2285945706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d8eef3510 0x555d8ef0c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.447+0000 7fcb2a818740 1 --2- 172.21.15.159:0/2285945706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d8eee8c50 0x555d8eeed600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.447+0000 7fcb2a818740 1 --2- 172.21.15.159:0/2285945706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d8eee8330 0x555d8eeed0c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.447+0000 7fcb06a00640 0 client.14949 ms_handle_reset on v2:172.21.15.189:6836/1030825585 2024-09-19T07:33:13.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.447+0000 7fcb2a818740 1 -- 172.21.15.159:0/2285945706 >> 172.21.15.159:0/2285945706 conn(0x555d8eee48c0 msgr2=0x555d8eeeef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:13.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.448+0000 7fcb2a818740 1 -- 172.21.15.159:0/2285945706 shutdown_connections 2024-09-19T07:33:13.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.448+0000 7fcb2a818740 1 -- 172.21.15.159:0/2285945706 wait complete. 2024-09-19T07:33:13.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.448+0000 7fcb2a818740 20 client.14949 trim_cache size 0 max 16384 2024-09-19T07:33:13.458 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-19T07:33:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "c9d18048-7657-11ef-bceb-c7b262605968", 2024-09-19T07:33:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 2, 2024-09-19T07:33:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 24506, 2024-09-19T07:33:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-09-19T07:33:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-09-19T07:33:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-09-19T07:33:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 82.562445953999998, 2024-09-19T07:33:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 25, 2024-09-19T07:33:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 100, 2024-09-19T07:33:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 98, 2024-09-19T07:33:13.458 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 157.57843505899999 2024-09-19T07:33:13.458 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-19T07:33:13.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.448+0000 7fcb27e00640 1 -- 172.21.15.159:0/3460081727 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcb080a3350 msgr2=0x7fcb080a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:13.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.448+0000 7fcb27e00640 1 --2- 172.21.15.159:0/3460081727 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcb080a3350 0x7fcb080a5810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400e9c0 tx=0x7fcb140d3830 comp rx=0 tx=0).stop 2024-09-19T07:33:13.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.448+0000 7fcb27e00640 1 -- 172.21.15.159:0/3460081727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb20074620 msgr2=0x7fcb2019a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:13.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.448+0000 7fcb27e00640 1 --2- 172.21.15.159:0/3460081727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb20074620 0x7fcb2019a660 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400e9f0 tx=0x7fcb1400eeb0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.448+0000 7fcb27e00640 1 -- 172.21.15.159:0/3460081727 shutdown_connections 2024-09-19T07:33:13.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.448+0000 7fcb27e00640 1 --2- 172.21.15.159:0/3460081727 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcb080a3350 0x7fcb080a5810 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.448+0000 7fcb27e00640 1 --2- 172.21.15.159:0/3460081727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb2010e3c0 0x7fcb2019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.448+0000 7fcb27e00640 1 --2- 172.21.15.159:0/3460081727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb20074f40 0x7fcb2019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.448+0000 7fcb27e00640 1 --2- 172.21.15.159:0/3460081727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb20074620 0x7fcb2019a660 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:13.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.449+0000 7fcb27e00640 1 -- 172.21.15.159:0/3460081727 >> 172.21.15.159:0/3460081727 conn(0x7fcb200fd8c0 msgr2=0x7fcb200fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:13.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.449+0000 7fcb27e00640 1 -- 172.21.15.159:0/3460081727 shutdown_connections 2024-09-19T07:33:13.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:13.449+0000 7fcb27e00640 1 -- 172.21.15.159:0/3460081727 wait complete. 2024-09-19T07:33:14.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:13 smithi203 ceph-mon[35335]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:33:14.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:13 smithi189 ceph-mon[34851]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:33:14.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:13 smithi159 ceph-mon[30472]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:33:14.290 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T07:33:14.662 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:15.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.272+0000 7f1e05c00640 1 -- 172.21.15.159:0/3608285066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e00109140 msgr2=0x7f1e00110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:15.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.272+0000 7f1e05c00640 1 --2- 172.21.15.159:0/3608285066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e00109140 0x7f1e00110cc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009be0 tx=0x7f1df802a6d0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.272+0000 7f1e05c00640 1 -- 172.21.15.159:0/3608285066 shutdown_connections 2024-09-19T07:33:15.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.272+0000 7f1e05c00640 1 --2- 172.21.15.159:0/3608285066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e00109140 0x7f1e00110cc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.272+0000 7f1e05c00640 1 --2- 172.21.15.159:0/3608285066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e001049c0 0x7f1e00108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.272+0000 7f1e05c00640 1 --2- 172.21.15.159:0/3608285066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e00104010 0x7f1e001043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.272+0000 7f1e05c00640 1 -- 172.21.15.159:0/3608285066 >> 172.21.15.159:0/3608285066 conn(0x7f1e000ffa50 msgr2=0x7f1e00101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:15.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.272+0000 7f1e05c00640 1 -- 172.21.15.159:0/3608285066 shutdown_connections 2024-09-19T07:33:15.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.272+0000 7f1e05c00640 1 -- 172.21.15.159:0/3608285066 wait complete. 2024-09-19T07:33:15.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.274+0000 7f1e05c00640 1 Processor -- start 2024-09-19T07:33:15.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.274+0000 7f1e05c00640 1 -- start start 2024-09-19T07:33:15.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.274+0000 7f1e05c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e00104010 0x7f1e0019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.274+0000 7f1e05c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e001049c0 0x7f1e0019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.274+0000 7f1e05c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e00109140 0x7f1e0019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.274+0000 7f1e05c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00114100 con 0x7f1e00104010 2024-09-19T07:33:15.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.274+0000 7f1e05c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00113f80 con 0x7f1e001049c0 2024-09-19T07:33:15.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.274+0000 7f1e05c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00114280 con 0x7f1e00109140 2024-09-19T07:33:15.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.275+0000 7f1dfea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e00104010 0x7f1e0019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.275+0000 7f1dfea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e00104010 0x7f1e0019a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:57078/0 (socket says 172.21.15.159:57078) 2024-09-19T07:33:15.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.275+0000 7f1dfea00640 1 -- 172.21.15.159:0/1764402966 learned_addr learned my addr 172.21.15.159:0/1764402966 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:15.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.275+0000 7f1dfe000640 1 --2- 172.21.15.159:0/1764402966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e001049c0 0x7f1e0019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.275+0000 7f1dff400640 1 --2- 172.21.15.159:0/1764402966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e00109140 0x7f1e0019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.275+0000 7f1dfea00640 1 -- 172.21.15.159:0/1764402966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e00109140 msgr2=0x7f1e0019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:15.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.275+0000 7f1dfea00640 1 --2- 172.21.15.159:0/1764402966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e00109140 0x7f1e0019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.275+0000 7f1dfea00640 1 -- 172.21.15.159:0/1764402966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e001049c0 msgr2=0x7f1e0019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:15.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.275+0000 7f1dfea00640 1 --2- 172.21.15.159:0/1764402966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e001049c0 0x7f1e0019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.275+0000 7f1dfea00640 1 -- 172.21.15.159:0/1764402966 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0019f6b0 con 0x7f1e00104010 2024-09-19T07:33:15.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.275+0000 7f1dff400640 1 --2- 172.21.15.159:0/1764402966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e00109140 0x7f1e0019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:33:15.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.276+0000 7f1dfea00640 1 --2- 172.21.15.159:0/1764402966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e00104010 0x7f1e0019a700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1df000e9c0 tx=0x7f1df000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:15.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.276+0000 7f1df7400640 1 -- 172.21.15.159:0/1764402966 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df000cde0 con 0x7f1e00104010 2024-09-19T07:33:15.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.276+0000 7f1e05c00640 1 -- 172.21.15.159:0/1764402966 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e0019f9a0 con 0x7f1e00104010 2024-09-19T07:33:15.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.276+0000 7f1df7400640 1 -- 172.21.15.159:0/1764402966 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1df00035f0 con 0x7f1e00104010 2024-09-19T07:33:15.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.276+0000 7f1e05c00640 1 -- 172.21.15.159:0/1764402966 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e001a7280 con 0x7f1e00104010 2024-09-19T07:33:15.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.276+0000 7f1df7400640 1 -- 172.21.15.159:0/1764402966 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0018630 con 0x7f1e00104010 2024-09-19T07:33:15.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.279+0000 7f1df7400640 1 -- 172.21.15.159:0/1764402966 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1df00037a0 con 0x7f1e00104010 2024-09-19T07:33:15.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.279+0000 7f1e0869f740 1 Processor -- start 2024-09-19T07:33:15.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.279+0000 7f1e0869f740 1 -- start start 2024-09-19T07:33:15.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.279+0000 7f1e0869f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92e330 0x55d4ad92e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.279+0000 7f1e0869f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92ec50 0x55d4ad938fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.280+0000 7f1e0869f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad939510 0x55d4ad93b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.280+0000 7f1e0869f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d4ad8940b0 con 0x55d4ad92e330 2024-09-19T07:33:15.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.280+0000 7f1e0869f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d4ad83bab0 con 0x55d4ad939510 2024-09-19T07:33:15.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.280+0000 7f1e0869f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d4ad890070 con 0x55d4ad92ec50 2024-09-19T07:33:15.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.280+0000 7f1dff400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92e330 0x55d4ad92e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.280+0000 7f1dff400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92e330 0x55d4ad92e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:57094/0 (socket says 172.21.15.159:57094) 2024-09-19T07:33:15.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.280+0000 7f1dff400640 1 -- 172.21.15.159:0/1835182494 learned_addr learned my addr 172.21.15.159:0/1835182494 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:15.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.280+0000 7f1dfe000640 1 --2- 172.21.15.159:0/1835182494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad939510 0x55d4ad93b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.280+0000 7f1dfea00640 1 --2- 172.21.15.159:0/1835182494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92ec50 0x55d4ad938fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.280+0000 7f1dff400640 1 -- 172.21.15.159:0/1835182494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92ec50 msgr2=0x55d4ad938fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:15.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.280+0000 7f1dff400640 1 --2- 172.21.15.159:0/1835182494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92ec50 0x55d4ad938fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.280+0000 7f1dff400640 1 -- 172.21.15.159:0/1835182494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad939510 msgr2=0x55d4ad93b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:15.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.280+0000 7f1dff400640 1 --2- 172.21.15.159:0/1835182494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad939510 0x55d4ad93b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.280+0000 7f1dff400640 1 -- 172.21.15.159:0/1835182494 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4ad8484a0 con 0x55d4ad92e330 2024-09-19T07:33:15.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.280+0000 7f1dfe000640 1 --2- 172.21.15.159:0/1835182494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad939510 0x55d4ad93b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:15.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.281+0000 7f1dff400640 1 --2- 172.21.15.159:0/1835182494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92e330 0x55d4ad92e710 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1df8033000 tx=0x7f1df8030a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118bdee7a57f392 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:15.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.281+0000 7f1df7400640 1 --2- 172.21.15.159:0/1764402966 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1de40a32d0 0x7f1de40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.281+0000 7f1debe00640 1 -- 172.21.15.159:0/1835182494 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8041030 con 0x55d4ad92e330 2024-09-19T07:33:15.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.281+0000 7f1debe00640 1 -- 172.21.15.159:0/1835182494 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1df8037040 con 0x55d4ad92e330 2024-09-19T07:33:15.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.281+0000 7f1debe00640 1 -- 172.21.15.159:0/1835182494 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8036040 con 0x55d4ad92e330 2024-09-19T07:33:15.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.281+0000 7f1df7400640 1 -- 172.21.15.159:0/1764402966 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f1df00d01e0 con 0x7f1e00104010 2024-09-19T07:33:15.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.281+0000 7f1dfea00640 1 --2- 172.21.15.159:0/1764402966 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1de40a32d0 0x7f1de40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.282+0000 7f1dfea00640 1 --2- 172.21.15.159:0/1764402966 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1de40a32d0 0x7f1de40a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1df0098c60 tx=0x7f1df00d3c90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:15.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.282+0000 7f1e0869f740 1 -- 172.21.15.159:0/1835182494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92e330 msgr2=0x55d4ad92e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:15.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.282+0000 7f1e0869f740 1 --2- 172.21.15.159:0/1835182494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92e330 0x55d4ad92e710 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1df8033000 tx=0x7f1df8030a30 comp rx=0 tx=0).stop 2024-09-19T07:33:15.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.282+0000 7f1e0869f740 1 -- 172.21.15.159:0/1835182494 shutdown_connections 2024-09-19T07:33:15.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.282+0000 7f1e0869f740 1 --2- 172.21.15.159:0/1835182494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad939510 0x55d4ad93b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.282+0000 7f1e0869f740 1 --2- 172.21.15.159:0/1835182494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92ec50 0x55d4ad938fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.282+0000 7f1e0869f740 1 --2- 172.21.15.159:0/1835182494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92e330 0x55d4ad92e710 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.282+0000 7f1e0869f740 1 -- 172.21.15.159:0/1835182494 >> 172.21.15.159:0/1835182494 conn(0x55d4ad92a8c0 msgr2=0x55d4ad92aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:15.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.282+0000 7f1e0869f740 1 -- 172.21.15.159:0/1835182494 shutdown_connections 2024-09-19T07:33:15.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.282+0000 7f1e0869f740 1 -- 172.21.15.159:0/1835182494 wait complete. 2024-09-19T07:33:15.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.283+0000 7f1e0869f740 1 -- start start 2024-09-19T07:33:15.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.283+0000 7f1e0869f740 1 Processor -- start 2024-09-19T07:33:15.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.283+0000 7f1e0869f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad92e330 0x55d4ad936530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.283+0000 7f1e0869f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92ec50 0x55d4ad936a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.283+0000 7f1e0869f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad939510 0x55d4ad936fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.283+0000 7f1e0869f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d4ad93c060 con 0x55d4ad939510 2024-09-19T07:33:15.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.283+0000 7f1e0869f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d4ad81c920 con 0x55d4ad92e330 2024-09-19T07:33:15.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.283+0000 7f1e0869f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d4ad93c1e0 con 0x55d4ad92ec50 2024-09-19T07:33:15.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.284+0000 7f1e0869f740 10 client.0 _ll_register_callbacks cb 0x55d4ad8408a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:33:15.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.284+0000 7f1dff400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad939510 0x55d4ad936fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.284+0000 7f1dff400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad939510 0x55d4ad936fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:57098/0 (socket says 172.21.15.159:57098) 2024-09-19T07:33:15.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.284+0000 7f1dff400640 1 -- 172.21.15.159:0/711836718 learned_addr learned my addr 172.21.15.159:0/711836718 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:15.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.284+0000 7f1dfe000640 1 --2- 172.21.15.159:0/711836718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92ec50 0x55d4ad936a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.284+0000 7f1dff400640 1 --2- 172.21.15.159:0/711836718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad92e330 0x55d4ad936530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.284+0000 7f1dff400640 1 -- 172.21.15.159:0/711836718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92ec50 msgr2=0x55d4ad936a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:15.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.284+0000 7f1dff400640 1 --2- 172.21.15.159:0/711836718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92ec50 0x55d4ad936a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.284+0000 7f1dff400640 1 -- 172.21.15.159:0/711836718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad92e330 msgr2=0x55d4ad936530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:15.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.284+0000 7f1dff400640 1 --2- 172.21.15.159:0/711836718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad92e330 0x55d4ad936530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.284+0000 7f1dff400640 1 -- 172.21.15.159:0/711836718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4ad6052b0 con 0x55d4ad939510 2024-09-19T07:33:15.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.285+0000 7f1dff400640 1 --2- 172.21.15.159:0/711836718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad939510 0x55d4ad936fb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1df8034040 tx=0x7f1df8003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e10be00a2161b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:15.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.285+0000 7f1de8c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-19T07:33:15.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.285+0000 7f1de8c00640 1 -- 172.21.15.159:0/711836718 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8041070 con 0x55d4ad939510 2024-09-19T07:33:15.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.285+0000 7f1de8c00640 1 -- 172.21.15.159:0/711836718 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f1df804c030 con 0x55d4ad939510 2024-09-19T07:33:15.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.285+0000 7f1e0869f740 1 -- 172.21.15.159:0/711836718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1df8037040 con 0x55d4ad939510 2024-09-19T07:33:15.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.285+0000 7f1de8c00640 1 -- 172.21.15.159:0/711836718 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1df80307e0 con 0x55d4ad939510 2024-09-19T07:33:15.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.285+0000 7f1e0869f740 10 client.14961 fetch_fsmap learned FSMap version 25 2024-09-19T07:33:15.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.285+0000 7f1e0869f740 1 -- 172.21.15.159:0/711836718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d4ad947db0 con 0x55d4ad939510 2024-09-19T07:33:15.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.285+0000 7f1de8c00640 1 -- 172.21.15.159:0/711836718 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1df8037040 con 0x55d4ad939510 2024-09-19T07:33:15.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.286+0000 7f1de8c00640 1 -- 172.21.15.159:0/711836718 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1df803c040 con 0x55d4ad939510 2024-09-19T07:33:15.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.286+0000 7f1e0869f740 10 client.14961 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T07:33:15.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.286+0000 7f1e0869f740 10 client.14961 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T07:33:15.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.286+0000 7f1e0869f740 20 client.14961 populate_metadata read hostname 'smithi159' 2024-09-19T07:33:15.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.286+0000 7f1e0869f740 1 --2- 172.21.15.159:0/711836718 >> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] conn(0x55d4ad949ec0 0x55d4ad94b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.286+0000 7f1e0869f740 4 client.14961 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:33:15.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.286+0000 7f1e0869f740 1 -- 172.21.15.159:0/711836718 --> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4ad8a1620 con 0x55d4ad949ec0 2024-09-19T07:33:15.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.286+0000 7f1dfe000640 1 --2- 172.21.15.159:0/711836718 >> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] conn(0x55d4ad949ec0 0x55d4ad94b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.287+0000 7f1dfe000640 1 --2- 172.21.15.159:0/711836718 >> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] conn(0x55d4ad949ec0 0x55d4ad94b330 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4600521f661ee85c server_cookie=69d890aeeaa51467 in_seq=0 out_seq=0 2024-09-19T07:33:15.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.287+0000 7f1de8c00640 10 client.14961 ms_handle_connect on v2:172.21.15.203:6838/3862781462 2024-09-19T07:33:15.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.287+0000 7f1de8c00640 1 -- 172.21.15.159:0/711836718 <== mds.1 v2:172.21.15.203:6838/3862781462 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d4ad8a1620 con 0x55d4ad949ec0 2024-09-19T07:33:15.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.287+0000 7f1de8c00640 10 client.14961 handle_command_reply: tid=0 2024-09-19T07:33:15.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.288+0000 7f1e0869f740 1 client.14961 shutdown 2024-09-19T07:33:15.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.288+0000 7f1e0869f740 1 -- 172.21.15.159:0/711836718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad939510 msgr2=0x55d4ad936fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:15.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.288+0000 7f1e0869f740 1 --2- 172.21.15.159:0/711836718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad939510 0x55d4ad936fb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1df8034040 tx=0x7f1df8003d20 comp rx=0 tx=0).stop 2024-09-19T07:33:15.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.288+0000 7f1e0869f740 1 -- 172.21.15.159:0/711836718 shutdown_connections 2024-09-19T07:33:15.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.288+0000 7f1e0869f740 1 --2- 172.21.15.159:0/711836718 >> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] conn(0x55d4ad949ec0 0x55d4ad94b330 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.288+0000 7f1e0869f740 1 --2- 172.21.15.159:0/711836718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad939510 0x55d4ad936fb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.288+0000 7f1e0869f740 1 --2- 172.21.15.159:0/711836718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92ec50 0x55d4ad936a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.288+0000 7f1e0869f740 1 --2- 172.21.15.159:0/711836718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad92e330 0x55d4ad936530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.288+0000 7f1de8c00640 0 client.14961 ms_handle_reset on v2:172.21.15.203:6838/3862781462 2024-09-19T07:33:15.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.288+0000 7f1e0869f740 1 -- 172.21.15.159:0/711836718 >> 172.21.15.159:0/711836718 conn(0x55d4ad92a8c0 msgr2=0x55d4ad9305a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:15.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.288+0000 7f1e0869f740 1 -- 172.21.15.159:0/711836718 shutdown_connections 2024-09-19T07:33:15.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.288+0000 7f1e0869f740 1 -- 172.21.15.159:0/711836718 wait complete. 2024-09-19T07:33:15.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.288+0000 7f1e0869f740 20 client.14961 trim_cache size 0 max 16384 2024-09-19T07:33:15.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.307+0000 7f1e0869f740 1 Processor -- start 2024-09-19T07:33:15.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.307+0000 7f1e0869f740 1 -- start start 2024-09-19T07:33:15.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.307+0000 7f1e0869f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92e330 0x55d4ad930d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.307+0000 7f1e0869f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92ec50 0x55d4ad936e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.307+0000 7f1e0869f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad939510 0x55d4ad953610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.307+0000 7f1e0869f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d4ad949670 con 0x55d4ad92ec50 2024-09-19T07:33:15.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.307+0000 7f1e0869f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d4ad949d40 con 0x55d4ad939510 2024-09-19T07:33:15.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.307+0000 7f1e0869f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d4ad94a410 con 0x55d4ad92e330 2024-09-19T07:33:15.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.307+0000 7f1dfe000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92ec50 0x55d4ad936e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.307+0000 7f1dfe000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92ec50 0x55d4ad936e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:57110/0 (socket says 172.21.15.159:57110) 2024-09-19T07:33:15.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.307+0000 7f1dfe000640 1 -- 172.21.15.159:0/2932190827 learned_addr learned my addr 172.21.15.159:0/2932190827 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:15.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.308+0000 7f1dff400640 1 --2- 172.21.15.159:0/2932190827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92e330 0x55d4ad930d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.308+0000 7f1dff400640 1 --2- 172.21.15.159:0/2932190827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad939510 0x55d4ad953610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.308+0000 7f1dfe000640 1 -- 172.21.15.159:0/2932190827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92e330 msgr2=0x55d4ad930d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:15.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.308+0000 7f1dfe000640 1 --2- 172.21.15.159:0/2932190827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92e330 0x55d4ad930d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.308+0000 7f1dfe000640 1 -- 172.21.15.159:0/2932190827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad939510 msgr2=0x55d4ad953610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:15.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.308+0000 7f1dfe000640 1 --2- 172.21.15.159:0/2932190827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad939510 0x55d4ad953610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.308+0000 7f1dfe000640 1 -- 172.21.15.159:0/2932190827 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4ad934a80 con 0x55d4ad92ec50 2024-09-19T07:33:15.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.308+0000 7f1dff400640 1 --2- 172.21.15.159:0/2932190827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad939510 0x55d4ad953610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:33:15.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.308+0000 7f1dfe000640 1 --2- 172.21.15.159:0/2932190827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92ec50 0x55d4ad936e00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55d4ad935dc0 tx=0x7f1dec005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43f1ef0e7170abe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:15.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.308+0000 7f1deb400640 1 -- 172.21.15.159:0/2932190827 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec027440 con 0x55d4ad92ec50 2024-09-19T07:33:15.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.308+0000 7f1deb400640 1 -- 172.21.15.159:0/2932190827 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dec027ac0 con 0x55d4ad92ec50 2024-09-19T07:33:15.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.308+0000 7f1deb400640 1 -- 172.21.15.159:0/2932190827 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec010a40 con 0x55d4ad92ec50 2024-09-19T07:33:15.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.309+0000 7f1e0869f740 1 -- 172.21.15.159:0/2932190827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92ec50 msgr2=0x55d4ad936e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:15.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.309+0000 7f1e0869f740 1 --2- 172.21.15.159:0/2932190827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92ec50 0x55d4ad936e00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55d4ad935dc0 tx=0x7f1dec005f90 comp rx=0 tx=0).stop 2024-09-19T07:33:15.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.309+0000 7f1e0869f740 1 -- 172.21.15.159:0/2932190827 shutdown_connections 2024-09-19T07:33:15.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.309+0000 7f1e0869f740 1 --2- 172.21.15.159:0/2932190827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad939510 0x55d4ad953610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.309+0000 7f1e0869f740 1 --2- 172.21.15.159:0/2932190827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92ec50 0x55d4ad936e00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.309+0000 7f1e0869f740 1 --2- 172.21.15.159:0/2932190827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92e330 0x55d4ad930d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.309+0000 7f1e0869f740 1 -- 172.21.15.159:0/2932190827 >> 172.21.15.159:0/2932190827 conn(0x55d4ad92a8c0 msgr2=0x55d4ad9385b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:15.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.309+0000 7f1e0869f740 1 -- 172.21.15.159:0/2932190827 shutdown_connections 2024-09-19T07:33:15.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.309+0000 7f1e0869f740 1 -- 172.21.15.159:0/2932190827 wait complete. 2024-09-19T07:33:15.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.310+0000 7f1e0869f740 1 -- start start 2024-09-19T07:33:15.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.310+0000 7f1e0869f740 1 Processor -- start 2024-09-19T07:33:15.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.310+0000 7f1e0869f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92e330 0x55d4ad953990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.310+0000 7f1e0869f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92ec50 0x55d4ad953ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.310+0000 7f1e0869f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad939510 0x55d4ad94e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.310+0000 7f1e0869f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d4ad94b1b0 con 0x55d4ad92e330 2024-09-19T07:33:15.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.310+0000 7f1e0869f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d4ad94aae0 con 0x55d4ad939510 2024-09-19T07:33:15.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.310+0000 7f1e0869f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d4ad94b880 con 0x55d4ad92ec50 2024-09-19T07:33:15.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.310+0000 7f1e0869f740 10 client.0 _ll_register_callbacks cb 0x7f1df0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:33:15.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.311+0000 7f1dfe000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92ec50 0x55d4ad953ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.311+0000 7f1dfe000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92ec50 0x55d4ad953ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:43144/0 (socket says 172.21.15.159:43144) 2024-09-19T07:33:15.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.311+0000 7f1dfe000640 1 -- 172.21.15.159:0/2231287052 learned_addr learned my addr 172.21.15.159:0/2231287052 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:15.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.311+0000 7f1dff400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92e330 0x55d4ad953990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.311+0000 7f1dff400640 1 --2- 172.21.15.159:0/2231287052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad939510 0x55d4ad94e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.311+0000 7f1dfe000640 1 -- 172.21.15.159:0/2231287052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad939510 msgr2=0x55d4ad94e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:15.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.311+0000 7f1dfe000640 1 --2- 172.21.15.159:0/2231287052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad939510 0x55d4ad94e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.311+0000 7f1dfe000640 1 -- 172.21.15.159:0/2231287052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92e330 msgr2=0x55d4ad953990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:15.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.311+0000 7f1dfe000640 1 --2- 172.21.15.159:0/2231287052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92e330 0x55d4ad953990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.311+0000 7f1dfe000640 1 -- 172.21.15.159:0/2231287052 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4ad94ef70 con 0x55d4ad92ec50 2024-09-19T07:33:15.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.311+0000 7f1dff400640 1 --2- 172.21.15.159:0/2231287052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92e330 0x55d4ad953990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:15.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.311+0000 7f1dfe000640 1 --2- 172.21.15.159:0/2231287052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92ec50 0x55d4ad953ed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009dd0 tx=0x7f1dec0106e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ee24736f7da7eb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:15.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.311+0000 7f1de8c00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-19T07:33:15.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.312+0000 7f1de8c00640 1 -- 172.21.15.159:0/2231287052 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec02c070 con 0x55d4ad92ec50 2024-09-19T07:33:15.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.312+0000 7f1de8c00640 1 -- 172.21.15.159:0/2231287052 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f1dec004220 con 0x55d4ad92ec50 2024-09-19T07:33:15.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.312+0000 7f1e0869f740 1 -- 172.21.15.159:0/2231287052 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dec027ac0 con 0x55d4ad92ec50 2024-09-19T07:33:15.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.312+0000 7f1de8c00640 1 -- 172.21.15.159:0/2231287052 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1dec004420 con 0x55d4ad92ec50 2024-09-19T07:33:15.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.312+0000 7f1e0869f740 10 client.25015 fetch_fsmap learned FSMap version 25 2024-09-19T07:33:15.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.312+0000 7f1e0869f740 1 -- 172.21.15.159:0/2231287052 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d4ad9b52a0 con 0x55d4ad92ec50 2024-09-19T07:33:15.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.312+0000 7f1de8c00640 1 -- 172.21.15.159:0/2231287052 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dec027ac0 con 0x55d4ad92ec50 2024-09-19T07:33:15.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.312+0000 7f1de8c00640 1 -- 172.21.15.159:0/2231287052 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1dec025020 con 0x55d4ad92ec50 2024-09-19T07:33:15.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.312+0000 7f1e0869f740 10 client.25015 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T07:33:15.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.312+0000 7f1e0869f740 10 client.25015 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T07:33:15.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.312+0000 7f1e0869f740 20 client.25015 populate_metadata read hostname 'smithi159' 2024-09-19T07:33:15.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.313+0000 7f1e0869f740 1 --2- 172.21.15.159:0/2231287052 >> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] conn(0x55d4ad9b9b20 0x55d4ad9b8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:15.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.313+0000 7f1e0869f740 4 client.25015 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:33:15.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.313+0000 7f1e0869f740 1 -- 172.21.15.159:0/2231287052 --> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] -- command(tid 0: {"prefix": "status"}) -- 0x55d4ad89bb30 con 0x55d4ad9b9b20 2024-09-19T07:33:15.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.313+0000 7f1dff400640 1 --2- 172.21.15.159:0/2231287052 >> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] conn(0x55d4ad9b9b20 0x55d4ad9b8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:15.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.313+0000 7f1dff400640 1 --2- 172.21.15.159:0/2231287052 >> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] conn(0x55d4ad9b9b20 0x55d4ad9b8830 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a8f07a9fe4460cbb server_cookie=48a6b786501befda in_seq=0 out_seq=0 2024-09-19T07:33:15.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.313+0000 7f1de8c00640 10 client.25015 ms_handle_connect on v2:172.21.15.203:6838/3862781462 2024-09-19T07:33:15.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.314+0000 7f1de8c00640 1 -- 172.21.15.159:0/2231287052 <== mds.1 v2:172.21.15.203:6838/3862781462 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55d4ad89bb30 con 0x55d4ad9b9b20 2024-09-19T07:33:15.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.314+0000 7f1de8c00640 10 client.25015 handle_command_reply: tid=0 2024-09-19T07:33:15.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.314+0000 7f1e0869f740 1 client.25015 shutdown 2024-09-19T07:33:15.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.314+0000 7f1e0869f740 1 -- 172.21.15.159:0/2231287052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92ec50 msgr2=0x55d4ad953ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:15.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.314+0000 7f1e0869f740 1 --2- 172.21.15.159:0/2231287052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92ec50 0x55d4ad953ed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009dd0 tx=0x7f1dec0106e0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.314+0000 7f1e0869f740 1 -- 172.21.15.159:0/2231287052 shutdown_connections 2024-09-19T07:33:15.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.314+0000 7f1e0869f740 1 --2- 172.21.15.159:0/2231287052 >> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] conn(0x55d4ad9b9b20 0x55d4ad9b8830 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.314+0000 7f1e0869f740 1 --2- 172.21.15.159:0/2231287052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d4ad939510 0x55d4ad94e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.314+0000 7f1e0869f740 1 --2- 172.21.15.159:0/2231287052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d4ad92ec50 0x55d4ad953ed0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.314+0000 7f1e0869f740 1 --2- 172.21.15.159:0/2231287052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ad92e330 0x55d4ad953990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.314+0000 7f1de8c00640 0 client.25015 ms_handle_reset on v2:172.21.15.203:6838/3862781462 2024-09-19T07:33:15.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.314+0000 7f1e0869f740 1 -- 172.21.15.159:0/2231287052 >> 172.21.15.159:0/2231287052 conn(0x55d4ad92a8c0 msgr2=0x55d4ad944430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:15.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.314+0000 7f1e0869f740 1 -- 172.21.15.159:0/2231287052 shutdown_connections 2024-09-19T07:33:15.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.314+0000 7f1e0869f740 1 -- 172.21.15.159:0/2231287052 wait complete. 2024-09-19T07:33:15.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.314+0000 7f1e0869f740 20 client.25015 trim_cache size 0 max 16384 2024-09-19T07:33:15.319 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-19T07:33:15.320 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "c9d18048-7657-11ef-bceb-c7b262605968", 2024-09-19T07:33:15.320 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 1, 2024-09-19T07:33:15.320 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 24514, 2024-09-19T07:33:15.320 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-09-19T07:33:15.320 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-09-19T07:33:15.320 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-09-19T07:33:15.320 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 85.438560914000007, 2024-09-19T07:33:15.320 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 25, 2024-09-19T07:33:15.320 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 100, 2024-09-19T07:33:15.320 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 98, 2024-09-19T07:33:15.320 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 156.79151375800001 2024-09-19T07:33:15.320 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-19T07:33:15.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.315+0000 7f1e05c00640 1 -- 172.21.15.159:0/1764402966 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1de40a32d0 msgr2=0x7f1de40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:15.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.315+0000 7f1e05c00640 1 --2- 172.21.15.159:0/1764402966 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1de40a32d0 0x7f1de40a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1df0098c60 tx=0x7f1df00d3c90 comp rx=0 tx=0).stop 2024-09-19T07:33:15.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.315+0000 7f1e05c00640 1 -- 172.21.15.159:0/1764402966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e00104010 msgr2=0x7f1e0019a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:15.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.315+0000 7f1e05c00640 1 --2- 172.21.15.159:0/1764402966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e00104010 0x7f1e0019a700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1df000e9c0 tx=0x7f1df000ee80 comp rx=0 tx=0).stop 2024-09-19T07:33:15.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.315+0000 7f1e05c00640 1 -- 172.21.15.159:0/1764402966 shutdown_connections 2024-09-19T07:33:15.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.315+0000 7f1e05c00640 1 --2- 172.21.15.159:0/1764402966 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f1de40a32d0 0x7f1de40a5790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.315+0000 7f1e05c00640 1 --2- 172.21.15.159:0/1764402966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e00109140 0x7f1e0019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.315+0000 7f1e05c00640 1 --2- 172.21.15.159:0/1764402966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e001049c0 0x7f1e0019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.315+0000 7f1e05c00640 1 --2- 172.21.15.159:0/1764402966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e00104010 0x7f1e0019a700 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:15.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.315+0000 7f1e05c00640 1 -- 172.21.15.159:0/1764402966 >> 172.21.15.159:0/1764402966 conn(0x7f1e000ffa50 msgr2=0x7f1e00100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:15.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.315+0000 7f1e05c00640 1 -- 172.21.15.159:0/1764402966 shutdown_connections 2024-09-19T07:33:15.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:15.315+0000 7f1e05c00640 1 -- 172.21.15.159:0/1764402966 wait complete. 2024-09-19T07:33:16.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:15 smithi189 ceph-mon[34851]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:16.078 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:15 smithi159 ceph-mon[30472]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:16.239 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T07:33:16.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:15 smithi203 ceph-mon[35335]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:16.607 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:17.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.149+0000 7f3dd5200640 1 -- 172.21.15.159:0/3426386872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dd0106f10 msgr2=0x7f3dd010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:17.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.149+0000 7f3dd5200640 1 --2- 172.21.15.159:0/3426386872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dd0106f10 0x7f3dd010ea90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009900 tx=0x7f3dc802a410 comp rx=0 tx=0).stop 2024-09-19T07:33:17.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.149+0000 7f3dd5200640 1 -- 172.21.15.159:0/3426386872 shutdown_connections 2024-09-19T07:33:17.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.149+0000 7f3dd5200640 1 --2- 172.21.15.159:0/3426386872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dd0106f10 0x7f3dd010ea90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.149+0000 7f3dd5200640 1 --2- 172.21.15.159:0/3426386872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dd0102790 0x7f3dd01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.149+0000 7f3dd5200640 1 --2- 172.21.15.159:0/3426386872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dd0101de0 0x7f3dd01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.149+0000 7f3dd5200640 1 -- 172.21.15.159:0/3426386872 >> 172.21.15.159:0/3426386872 conn(0x7f3dd00fd8c0 msgr2=0x7f3dd00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:17.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.149+0000 7f3dd5200640 1 -- 172.21.15.159:0/3426386872 shutdown_connections 2024-09-19T07:33:17.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.149+0000 7f3dd5200640 1 -- 172.21.15.159:0/3426386872 wait complete. 2024-09-19T07:33:17.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.150+0000 7f3dd5200640 1 Processor -- start 2024-09-19T07:33:17.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.150+0000 7f3dd5200640 1 -- start start 2024-09-19T07:33:17.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.150+0000 7f3dd5200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dd0101de0 0x7f3dd019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.150+0000 7f3dd5200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dd0102790 0x7f3dd019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.150+0000 7f3dd5200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dd0106f10 0x7f3dd019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.150+0000 7f3dd5200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd0111fc0 con 0x7f3dd0101de0 2024-09-19T07:33:17.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.150+0000 7f3dd5200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd0111e40 con 0x7f3dd0102790 2024-09-19T07:33:17.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.150+0000 7f3dd5200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd0112140 con 0x7f3dd0106f10 2024-09-19T07:33:17.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.151+0000 7f3dcd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dd0102790 0x7f3dd019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.151+0000 7f3dcde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dd0101de0 0x7f3dd019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.151+0000 7f3dcd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dd0102790 0x7f3dd019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:39056/0 (socket says 172.21.15.159:39056) 2024-09-19T07:33:17.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.151+0000 7f3dcd400640 1 -- 172.21.15.159:0/200048457 learned_addr learned my addr 172.21.15.159:0/200048457 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:17.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.151+0000 7f3dce800640 1 --2- 172.21.15.159:0/200048457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dd0106f10 0x7f3dd019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.151+0000 7f3dcd400640 1 -- 172.21.15.159:0/200048457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dd0106f10 msgr2=0x7f3dd019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:17.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.151+0000 7f3dcd400640 1 --2- 172.21.15.159:0/200048457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dd0106f10 0x7f3dd019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.151+0000 7f3dcd400640 1 -- 172.21.15.159:0/200048457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dd0101de0 msgr2=0x7f3dd019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:17.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.151+0000 7f3dcd400640 1 --2- 172.21.15.159:0/200048457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dd0101de0 0x7f3dd019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.151+0000 7f3dcd400640 1 -- 172.21.15.159:0/200048457 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd019f570 con 0x7f3dd0102790 2024-09-19T07:33:17.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.151+0000 7f3dcd400640 1 --2- 172.21.15.159:0/200048457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dd0102790 0x7f3dd019aba0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00e9f0 tx=0x7f3dbc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:17.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.152+0000 7f3dc6a00640 1 -- 172.21.15.159:0/200048457 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc00cd80 con 0x7f3dd0102790 2024-09-19T07:33:17.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.152+0000 7f3dc6a00640 1 -- 172.21.15.159:0/200048457 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dbc010910 con 0x7f3dd0102790 2024-09-19T07:33:17.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.152+0000 7f3dc6a00640 1 -- 172.21.15.159:0/200048457 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc018990 con 0x7f3dd0102790 2024-09-19T07:33:17.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.152+0000 7f3dd5200640 1 -- 172.21.15.159:0/200048457 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd019f860 con 0x7f3dd0102790 2024-09-19T07:33:17.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.152+0000 7f3dd5200640 1 -- 172.21.15.159:0/200048457 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd01a7140 con 0x7f3dd0102790 2024-09-19T07:33:17.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.155+0000 7f3dd7bd2740 1 Processor -- start 2024-09-19T07:33:17.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.155+0000 7f3dd7bd2740 1 -- start start 2024-09-19T07:33:17.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.155+0000 7f3dd7bd2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3d6330 0x5578ba3d6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.155+0000 7f3dd7bd2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6c50 0x5578ba3e0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.155+0000 7f3dd7bd2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3e1510 0x5578ba3e3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.155+0000 7f3dd7bd2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5578ba33c0b0 con 0x5578ba3d6c50 2024-09-19T07:33:17.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.155+0000 7f3dd7bd2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5578ba2e3ab0 con 0x5578ba3e1510 2024-09-19T07:33:17.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.155+0000 7f3dd7bd2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5578ba338070 con 0x5578ba3d6330 2024-09-19T07:33:17.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.155+0000 7f3dc6a00640 1 -- 172.21.15.159:0/200048457 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3dbc010ac0 con 0x7f3dd0102790 2024-09-19T07:33:17.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.155+0000 7f3dcde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6c50 0x5578ba3e0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.155+0000 7f3dce800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3d6330 0x5578ba3d6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.155+0000 7f3dcde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6c50 0x5578ba3e0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:57154/0 (socket says 172.21.15.159:57154) 2024-09-19T07:33:17.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.155+0000 7f3dcde00640 1 -- 172.21.15.159:0/3228596840 learned_addr learned my addr 172.21.15.159:0/3228596840 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:17.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.156+0000 7f3dcd400640 1 --2- 172.21.15.159:0/3228596840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3e1510 0x5578ba3e3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.156+0000 7f3dcde00640 1 -- 172.21.15.159:0/3228596840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3d6330 msgr2=0x5578ba3d6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:17.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.156+0000 7f3dcde00640 1 --2- 172.21.15.159:0/3228596840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3d6330 0x5578ba3d6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.156+0000 7f3dcde00640 1 -- 172.21.15.159:0/3228596840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3e1510 msgr2=0x5578ba3e3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:17.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.156+0000 7f3dcde00640 1 --2- 172.21.15.159:0/3228596840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3e1510 0x5578ba3e3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.156+0000 7f3dcde00640 1 -- 172.21.15.159:0/3228596840 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578ba2f04a0 con 0x5578ba3d6c50 2024-09-19T07:33:17.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.156+0000 7f3dce800640 1 --2- 172.21.15.159:0/3228596840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3d6330 0x5578ba3d6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:17.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.156+0000 7f3dcd400640 1 --2- 172.21.15.159:0/3228596840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3e1510 0x5578ba3e3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:33:17.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.156+0000 7f3dcde00640 1 --2- 172.21.15.159:0/3228596840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6c50 0x5578ba3e0fd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00114f0 tx=0x7f3dc00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d887f27c28854a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:17.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.156+0000 7f3dbb400640 1 -- 172.21.15.159:0/3228596840 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc001e070 con 0x5578ba3d6c50 2024-09-19T07:33:17.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.157+0000 7f3dbb400640 1 -- 172.21.15.159:0/3228596840 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dc000fb00 con 0x5578ba3d6c50 2024-09-19T07:33:17.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.157+0000 7f3dbb400640 1 -- 172.21.15.159:0/3228596840 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0019b90 con 0x5578ba3d6c50 2024-09-19T07:33:17.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.157+0000 7f3dc6a00640 1 --2- 172.21.15.159:0/200048457 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3db40a3350 0x7f3db40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.157+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/3228596840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6c50 msgr2=0x5578ba3e0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:17.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.157+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/3228596840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6c50 0x5578ba3e0fd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00114f0 tx=0x7f3dc00119b0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.157+0000 7f3dc6a00640 1 -- 172.21.15.159:0/200048457 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3dbc0d0080 con 0x7f3dd0102790 2024-09-19T07:33:17.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.157+0000 7f3dce800640 1 --2- 172.21.15.159:0/200048457 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3db40a3350 0x7f3db40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.157+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/3228596840 shutdown_connections 2024-09-19T07:33:17.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.157+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/3228596840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3e1510 0x5578ba3e3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.157+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/3228596840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6c50 0x5578ba3e0fd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.157+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/3228596840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3d6330 0x5578ba3d6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.157+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/3228596840 >> 172.21.15.159:0/3228596840 conn(0x5578ba3d28c0 msgr2=0x5578ba3d2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:17.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.158+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/3228596840 shutdown_connections 2024-09-19T07:33:17.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.158+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/3228596840 wait complete. 2024-09-19T07:33:17.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.158+0000 7f3dce800640 1 --2- 172.21.15.159:0/200048457 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3db40a3350 0x7f3db40a5810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8031550 tx=0x7f3dc80046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:17.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.158+0000 7f3dd7bd2740 1 -- start start 2024-09-19T07:33:17.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.158+0000 7f3dd7bd2740 1 Processor -- start 2024-09-19T07:33:17.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.159+0000 7f3dd7bd2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6330 0x5578ba3de6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.159+0000 7f3dd7bd2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3e1510 0x5578ba3dec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.159+0000 7f3dd7bd2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3df1b0 0x5578ba3ed590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.159+0000 7f3dd7bd2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5578ba3e40c0 con 0x5578ba3d6330 2024-09-19T07:33:17.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.159+0000 7f3dd7bd2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5578ba2c4920 con 0x5578ba3df1b0 2024-09-19T07:33:17.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.159+0000 7f3dd7bd2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5578ba3e4240 con 0x5578ba3e1510 2024-09-19T07:33:17.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.159+0000 7f3dd7bd2740 10 client.0 _ll_register_callbacks cb 0x5578ba2e88a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:33:17.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.159+0000 7f3dcde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6330 0x5578ba3de6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.159+0000 7f3dcde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6330 0x5578ba3de6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:57158/0 (socket says 172.21.15.159:57158) 2024-09-19T07:33:17.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.159+0000 7f3dcde00640 1 -- 172.21.15.159:0/2034937540 learned_addr learned my addr 172.21.15.159:0/2034937540 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:17.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.159+0000 7f3dcd400640 1 --2- 172.21.15.159:0/2034937540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3e1510 0x5578ba3dec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.159+0000 7f3dce800640 1 --2- 172.21.15.159:0/2034937540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3df1b0 0x5578ba3ed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.159+0000 7f3dcde00640 1 -- 172.21.15.159:0/2034937540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3e1510 msgr2=0x5578ba3dec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:17.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.159+0000 7f3dcde00640 1 --2- 172.21.15.159:0/2034937540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3e1510 0x5578ba3dec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.159+0000 7f3dcde00640 1 -- 172.21.15.159:0/2034937540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3df1b0 msgr2=0x5578ba3ed590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:17.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.159+0000 7f3dcde00640 1 --2- 172.21.15.159:0/2034937540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3df1b0 0x5578ba3ed590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.159+0000 7f3dcde00640 1 -- 172.21.15.159:0/2034937540 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578ba0ad2b0 con 0x5578ba3d6330 2024-09-19T07:33:17.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.160+0000 7f3dcde00640 1 --2- 172.21.15.159:0/2034937540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6330 0x5578ba3de6e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0011eb0 tx=0x7f3dc0019410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d72a6570e44f6c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:17.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.160+0000 7f3dafe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-19T07:33:17.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.160+0000 7f3dafe00640 1 -- 172.21.15.159:0/2034937540 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc001e040 con 0x5578ba3d6330 2024-09-19T07:33:17.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.160+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/2034937540 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dc000fb00 con 0x5578ba3d6330 2024-09-19T07:33:17.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.160+0000 7f3dafe00640 1 -- 172.21.15.159:0/2034937540 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3dc0004a40 con 0x5578ba3d6330 2024-09-19T07:33:17.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.160+0000 7f3dafe00640 1 -- 172.21.15.159:0/2034937540 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3dc0004ca0 con 0x5578ba3d6330 2024-09-19T07:33:17.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.160+0000 7f3dd7bd2740 10 client.14979 fetch_fsmap learned FSMap version 25 2024-09-19T07:33:17.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.160+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/2034937540 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5578ba3f03f0 con 0x5578ba3d6330 2024-09-19T07:33:17.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.161+0000 7f3dafe00640 1 -- 172.21.15.159:0/2034937540 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dc000fb00 con 0x5578ba3d6330 2024-09-19T07:33:17.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.161+0000 7f3dafe00640 1 -- 172.21.15.159:0/2034937540 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3dc0005160 con 0x5578ba3d6330 2024-09-19T07:33:17.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.161+0000 7f3dd7bd2740 10 client.14979 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T07:33:17.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.161+0000 7f3dd7bd2740 10 client.14979 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T07:33:17.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.161+0000 7f3dd7bd2740 20 client.14979 populate_metadata read hostname 'smithi159' 2024-09-19T07:33:17.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.161+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/2034937540 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x5578ba3f27f0 0x5578ba3f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.161+0000 7f3dd7bd2740 4 client.14979 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:33:17.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.161+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/2034937540 --> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578ba349620 con 0x5578ba3f27f0 2024-09-19T07:33:17.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.161+0000 7f3dcd400640 1 --2- 172.21.15.159:0/2034937540 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x5578ba3f27f0 0x5578ba3f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.162+0000 7f3dcd400640 1 --2- 172.21.15.159:0/2034937540 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x5578ba3f27f0 0x5578ba3f3c60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec968b46eccd5c57 server_cookie=b6ac4931f7448705 in_seq=0 out_seq=0 2024-09-19T07:33:17.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.162+0000 7f3dafe00640 10 client.14979 ms_handle_connect on v2:172.21.15.189:6838/2361818663 2024-09-19T07:33:17.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.163+0000 7f3dafe00640 1 -- 172.21.15.159:0/2034937540 <== mds.0 v2:172.21.15.189:6838/2361818663 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578ba349620 con 0x5578ba3f27f0 2024-09-19T07:33:17.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.163+0000 7f3dafe00640 10 client.14979 handle_command_reply: tid=0 2024-09-19T07:33:17.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.163+0000 7f3dd7bd2740 1 client.14979 shutdown 2024-09-19T07:33:17.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.163+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/2034937540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6330 msgr2=0x5578ba3de6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:17.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.163+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/2034937540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6330 0x5578ba3de6e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0011eb0 tx=0x7f3dc0019410 comp rx=0 tx=0).stop 2024-09-19T07:33:17.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.163+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/2034937540 shutdown_connections 2024-09-19T07:33:17.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.163+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/2034937540 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x5578ba3f27f0 0x5578ba3f3c60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.163+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/2034937540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3df1b0 0x5578ba3ed590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.163+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/2034937540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3e1510 0x5578ba3dec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.163+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/2034937540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6330 0x5578ba3de6e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.163+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/2034937540 >> 172.21.15.159:0/2034937540 conn(0x5578ba3d28c0 msgr2=0x5578ba3d85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:17.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.163+0000 7f3dafe00640 0 client.14979 ms_handle_reset on v2:172.21.15.189:6838/2361818663 2024-09-19T07:33:17.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.163+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/2034937540 shutdown_connections 2024-09-19T07:33:17.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.163+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/2034937540 wait complete. 2024-09-19T07:33:17.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.163+0000 7f3dd7bd2740 20 client.14979 trim_cache size 0 max 16384 2024-09-19T07:33:17.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.182+0000 7f3dd7bd2740 1 Processor -- start 2024-09-19T07:33:17.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.182+0000 7f3dd7bd2740 1 -- start start 2024-09-19T07:33:17.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.182+0000 7f3dd7bd2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6330 0x5578ba3ed500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.182+0000 7f3dd7bd2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3df1b0 0x5578ba3dcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.182+0000 7f3dd7bd2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3e1510 0x5578ba3dd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.182+0000 7f3dd7bd2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5578ba3f1fa0 con 0x5578ba3d6330 2024-09-19T07:33:17.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.182+0000 7f3dd7bd2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5578ba3f2670 con 0x5578ba3e1510 2024-09-19T07:33:17.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.182+0000 7f3dd7bd2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5578ba3f2d40 con 0x5578ba3df1b0 2024-09-19T07:33:17.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.182+0000 7f3dcde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6330 0x5578ba3ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.182+0000 7f3dcde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6330 0x5578ba3ed500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:57160/0 (socket says 172.21.15.159:57160) 2024-09-19T07:33:17.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.182+0000 7f3dcde00640 1 -- 172.21.15.159:0/5186387 learned_addr learned my addr 172.21.15.159:0/5186387 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:17.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.182+0000 7f3dce800640 1 --2- 172.21.15.159:0/5186387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3e1510 0x5578ba3dd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.183+0000 7f3dcd400640 1 --2- 172.21.15.159:0/5186387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3df1b0 0x5578ba3dcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.183+0000 7f3dcde00640 1 -- 172.21.15.159:0/5186387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3df1b0 msgr2=0x5578ba3dcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:17.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.183+0000 7f3dcde00640 1 --2- 172.21.15.159:0/5186387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3df1b0 0x5578ba3dcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.183+0000 7f3dcde00640 1 -- 172.21.15.159:0/5186387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3e1510 msgr2=0x5578ba3dd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:17.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.183+0000 7f3dcde00640 1 --2- 172.21.15.159:0/5186387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3e1510 0x5578ba3dd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.183+0000 7f3dcde00640 1 -- 172.21.15.159:0/5186387 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578ba3f3f90 con 0x5578ba3d6330 2024-09-19T07:33:17.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.183+0000 7f3dcde00640 1 --2- 172.21.15.159:0/5186387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6330 0x5578ba3ed500 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009560 tx=0x7f3dc0005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7594de352abf5cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:17.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.183+0000 7f3dbaa00640 1 -- 172.21.15.159:0/5186387 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc001e040 con 0x5578ba3d6330 2024-09-19T07:33:17.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.183+0000 7f3dbaa00640 1 -- 172.21.15.159:0/5186387 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dc0029070 con 0x5578ba3d6330 2024-09-19T07:33:17.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.183+0000 7f3dbaa00640 1 -- 172.21.15.159:0/5186387 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0019cc0 con 0x5578ba3d6330 2024-09-19T07:33:17.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.184+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/5186387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6330 msgr2=0x5578ba3ed500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:17.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.184+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/5186387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6330 0x5578ba3ed500 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009560 tx=0x7f3dc0005bf0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.184+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/5186387 shutdown_connections 2024-09-19T07:33:17.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.184+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/5186387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3e1510 0x5578ba3dd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.184+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/5186387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3df1b0 0x5578ba3dcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.184+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/5186387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3d6330 0x5578ba3ed500 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.184+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/5186387 >> 172.21.15.159:0/5186387 conn(0x5578ba3d28c0 msgr2=0x5578ba3d8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:17.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.184+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/5186387 shutdown_connections 2024-09-19T07:33:17.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.184+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/5186387 wait complete. 2024-09-19T07:33:17.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.184+0000 7f3dd7bd2740 1 -- start start 2024-09-19T07:33:17.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.185+0000 7f3dd7bd2740 1 Processor -- start 2024-09-19T07:33:17.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.185+0000 7f3dd7bd2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3d6330 0x5578ba3fc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.185+0000 7f3dd7bd2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3df1b0 0x5578ba3fc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.185+0000 7f3dd7bd2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3e1510 0x5578ba3db2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.185+0000 7f3dd7bd2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5578ba3f3ae0 con 0x5578ba3e1510 2024-09-19T07:33:17.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.185+0000 7f3dd7bd2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5578ba3f3410 con 0x5578ba3d6330 2024-09-19T07:33:17.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.185+0000 7f3dd7bd2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5578ba3f41b0 con 0x5578ba3df1b0 2024-09-19T07:33:17.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.185+0000 7f3dd7bd2740 10 client.0 _ll_register_callbacks cb 0x7f3dc0009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:33:17.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.185+0000 7f3dce800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3e1510 0x5578ba3db2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.185+0000 7f3dce800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3e1510 0x5578ba3db2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:57176/0 (socket says 172.21.15.159:57176) 2024-09-19T07:33:17.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.185+0000 7f3dce800640 1 -- 172.21.15.159:0/318184711 learned_addr learned my addr 172.21.15.159:0/318184711 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:17.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.185+0000 7f3dcd400640 1 --2- 172.21.15.159:0/318184711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3df1b0 0x5578ba3fc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.185+0000 7f3dcde00640 1 --2- 172.21.15.159:0/318184711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3d6330 0x5578ba3fc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.186+0000 7f3dce800640 1 -- 172.21.15.159:0/318184711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3df1b0 msgr2=0x5578ba3fc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:17.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.186+0000 7f3dce800640 1 --2- 172.21.15.159:0/318184711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3df1b0 0x5578ba3fc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.186+0000 7f3dce800640 1 -- 172.21.15.159:0/318184711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3d6330 msgr2=0x5578ba3fc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:17.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.186+0000 7f3dce800640 1 --2- 172.21.15.159:0/318184711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3d6330 0x5578ba3fc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.186+0000 7f3dce800640 1 -- 172.21.15.159:0/318184711 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578ba3dbb00 con 0x5578ba3e1510 2024-09-19T07:33:17.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.186+0000 7f3dcde00640 1 --2- 172.21.15.159:0/318184711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3d6330 0x5578ba3fc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:17.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.186+0000 7f3dcd400640 1 --2- 172.21.15.159:0/318184711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3df1b0 0x5578ba3fc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:17.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.186+0000 7f3dce800640 1 --2- 172.21.15.159:0/318184711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3e1510 0x5578ba3db2d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009900 tx=0x7f3dc8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c795631722e6c6dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:17.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.186+0000 7f3dafe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-19T07:33:17.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.186+0000 7f3dafe00640 1 -- 172.21.15.159:0/318184711 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8035040 con 0x5578ba3e1510 2024-09-19T07:33:17.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.186+0000 7f3dafe00640 1 -- 172.21.15.159:0/318184711 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3dc8031d60 con 0x5578ba3e1510 2024-09-19T07:33:17.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.186+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/318184711 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dc0029070 con 0x5578ba3e1510 2024-09-19T07:33:17.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.187+0000 7f3dafe00640 1 -- 172.21.15.159:0/318184711 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3dc80035f0 con 0x5578ba3e1510 2024-09-19T07:33:17.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.187+0000 7f3dd7bd2740 10 client.14985 fetch_fsmap learned FSMap version 25 2024-09-19T07:33:17.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.187+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/318184711 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5578ba3fbbd0 con 0x5578ba3e1510 2024-09-19T07:33:17.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.187+0000 7f3dafe00640 1 -- 172.21.15.159:0/318184711 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dc0029070 con 0x5578ba3e1510 2024-09-19T07:33:17.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.187+0000 7f3dafe00640 1 -- 172.21.15.159:0/318184711 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3dc803d930 con 0x5578ba3e1510 2024-09-19T07:33:17.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.187+0000 7f3dd7bd2740 10 client.14985 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T07:33:17.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.187+0000 7f3dd7bd2740 10 client.14985 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T07:33:17.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.187+0000 7f3dd7bd2740 20 client.14985 populate_metadata read hostname 'smithi159' 2024-09-19T07:33:17.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.187+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/318184711 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x5578ba461870 0x5578ba460580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:17.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.187+0000 7f3dd7bd2740 4 client.14985 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:33:17.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.187+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/318184711 --> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] -- command(tid 0: {"prefix": "status"}) -- 0x5578ba343b30 con 0x5578ba461870 2024-09-19T07:33:17.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.188+0000 7f3dcde00640 1 --2- 172.21.15.159:0/318184711 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x5578ba461870 0x5578ba460580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:17.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.188+0000 7f3dcde00640 1 --2- 172.21.15.159:0/318184711 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x5578ba461870 0x5578ba460580 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c306c81ad85ebfb2 server_cookie=f8dbaea5d9e33f63 in_seq=0 out_seq=0 2024-09-19T07:33:17.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.188+0000 7f3dafe00640 10 client.14985 ms_handle_connect on v2:172.21.15.189:6838/2361818663 2024-09-19T07:33:17.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.188+0000 7f3dafe00640 1 -- 172.21.15.159:0/318184711 <== mds.0 v2:172.21.15.189:6838/2361818663 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5578ba343b30 con 0x5578ba461870 2024-09-19T07:33:17.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.188+0000 7f3dafe00640 10 client.14985 handle_command_reply: tid=0 2024-09-19T07:33:17.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.188+0000 7f3dd7bd2740 1 client.14985 shutdown 2024-09-19T07:33:17.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.189+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/318184711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3e1510 msgr2=0x5578ba3db2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:17.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.189+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/318184711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3e1510 0x5578ba3db2d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009900 tx=0x7f3dc8009260 comp rx=0 tx=0).stop 2024-09-19T07:33:17.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.189+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/318184711 shutdown_connections 2024-09-19T07:33:17.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.189+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/318184711 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x5578ba461870 0x5578ba460580 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.189+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/318184711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578ba3e1510 0x5578ba3db2d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.189+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/318184711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578ba3df1b0 0x5578ba3fc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.189+0000 7f3dd7bd2740 1 --2- 172.21.15.159:0/318184711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578ba3d6330 0x5578ba3fc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.189+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/318184711 >> 172.21.15.159:0/318184711 conn(0x5578ba3d28c0 msgr2=0x5578ba3dcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:17.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.189+0000 7f3dafe00640 0 client.14985 ms_handle_reset on v2:172.21.15.189:6838/2361818663 2024-09-19T07:33:17.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.189+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/318184711 shutdown_connections 2024-09-19T07:33:17.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.189+0000 7f3dd7bd2740 1 -- 172.21.15.159:0/318184711 wait complete. 2024-09-19T07:33:17.291 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-19T07:33:17.291 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "c9d18048-7657-11ef-bceb-c7b262605968", 2024-09-19T07:33:17.291 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 0, 2024-09-19T07:33:17.291 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 24542, 2024-09-19T07:33:17.292 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-09-19T07:33:17.292 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-09-19T07:33:17.292 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-09-19T07:33:17.292 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 132.76120125400001, 2024-09-19T07:33:17.292 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 25, 2024-09-19T07:33:17.292 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 100, 2024-09-19T07:33:17.292 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 98, 2024-09-19T07:33:17.292 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 147.556064881 2024-09-19T07:33:17.292 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-19T07:33:17.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.189+0000 7f3dd7bd2740 20 client.14985 trim_cache size 0 max 16384 2024-09-19T07:33:17.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.189+0000 7f3dd5200640 1 -- 172.21.15.159:0/200048457 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3db40a3350 msgr2=0x7f3db40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:17.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.189+0000 7f3dd5200640 1 --2- 172.21.15.159:0/200048457 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3db40a3350 0x7f3db40a5810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8031550 tx=0x7f3dc80046c0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.189+0000 7f3dd5200640 1 -- 172.21.15.159:0/200048457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dd0102790 msgr2=0x7f3dd019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:17.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.189+0000 7f3dd5200640 1 --2- 172.21.15.159:0/200048457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dd0102790 0x7f3dd019aba0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00e9f0 tx=0x7f3dbc00eeb0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.190+0000 7f3dd5200640 1 -- 172.21.15.159:0/200048457 shutdown_connections 2024-09-19T07:33:17.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.190+0000 7f3dd5200640 1 --2- 172.21.15.159:0/200048457 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3db40a3350 0x7f3db40a5810 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.190+0000 7f3dd5200640 1 --2- 172.21.15.159:0/200048457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dd0106f10 0x7f3dd019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.190+0000 7f3dd5200640 1 --2- 172.21.15.159:0/200048457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dd0102790 0x7f3dd019aba0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.190+0000 7f3dd5200640 1 --2- 172.21.15.159:0/200048457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dd0101de0 0x7f3dd019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:17.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.190+0000 7f3dd5200640 1 -- 172.21.15.159:0/200048457 >> 172.21.15.159:0/200048457 conn(0x7f3dd00fd8c0 msgr2=0x7f3dd00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:17.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.190+0000 7f3dd5200640 1 -- 172.21.15.159:0/200048457 shutdown_connections 2024-09-19T07:33:17.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:17.190+0000 7f3dd5200640 1 -- 172.21.15.159:0/200048457 wait complete. 2024-09-19T07:33:18.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:17 smithi189 ceph-mon[34851]: pgmap v383: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:33:18.069 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:17 smithi159 ceph-mon[30472]: pgmap v383: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:33:18.088 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:17 smithi203 ceph-mon[35335]: pgmap v383: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:33:18.107 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T07:33:18.108 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T07:33:18.118 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi159.front.sepia.ceph.com 2024-09-19T07:33:18.119 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T07:33:18.489 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:19.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.117+0000 7fc984a00640 1 -- 172.21.15.159:0/3279323649 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9780319b0 con 0x7fc988100e90 2024-09-19T07:33:19.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.117+0000 7fc98ce00640 1 -- 172.21.15.159:0/3279323649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc988100e90 msgr2=0x7fc988101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:19.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.117+0000 7fc98ce00640 1 --2- 172.21.15.159:0/3279323649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc988100e90 0x7fc988101290 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc978009900 tx=0x7fc97802a3f0 comp rx=0 tx=0).stop 2024-09-19T07:33:19.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.117+0000 7fc98ce00640 1 -- 172.21.15.159:0/3279323649 shutdown_connections 2024-09-19T07:33:19.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.117+0000 7fc98ce00640 1 --2- 172.21.15.159:0/3279323649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc988100060 0x7fc988101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:19.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.117+0000 7fc98ce00640 1 --2- 172.21.15.159:0/3279323649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc988101860 0x7fc9880ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:19.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.117+0000 7fc98ce00640 1 --2- 172.21.15.159:0/3279323649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc988100e90 0x7fc988101290 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:19.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.117+0000 7fc98ce00640 1 -- 172.21.15.159:0/3279323649 >> 172.21.15.159:0/3279323649 conn(0x7fc9880fb610 msgr2=0x7fc9880fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:19.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.118+0000 7fc98ce00640 1 -- 172.21.15.159:0/3279323649 shutdown_connections 2024-09-19T07:33:19.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.118+0000 7fc98ce00640 1 -- 172.21.15.159:0/3279323649 wait complete. 2024-09-19T07:33:19.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.118+0000 7fc98ce00640 1 Processor -- start 2024-09-19T07:33:19.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.119+0000 7fc98ce00640 1 -- start start 2024-09-19T07:33:19.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.119+0000 7fc98ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc988100060 0x7fc98819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:19.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.119+0000 7fc98ce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc988100e90 0x7fc98819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:19.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.119+0000 7fc98ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc988101860 0x7fc9881a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:19.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.119+0000 7fc98ce00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc98810fa70 con 0x7fc988100060 2024-09-19T07:33:19.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.119+0000 7fc98ce00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc98810f8f0 con 0x7fc988101860 2024-09-19T07:33:19.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.119+0000 7fc98ce00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc98810fbf0 con 0x7fc988100e90 2024-09-19T07:33:19.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.119+0000 7fc985e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc988100060 0x7fc98819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:19.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.119+0000 7fc985e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc988100060 0x7fc98819a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:57196/0 (socket says 172.21.15.159:57196) 2024-09-19T07:33:19.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.119+0000 7fc985e00640 1 -- 172.21.15.159:0/2828607888 learned_addr learned my addr 172.21.15.159:0/2828607888 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:19.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.119+0000 7fc986800640 1 --2- 172.21.15.159:0/2828607888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc988101860 0x7fc9881a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:19.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.119+0000 7fc985400640 1 --2- 172.21.15.159:0/2828607888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc988100e90 0x7fc98819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:19.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.120+0000 7fc985e00640 1 -- 172.21.15.159:0/2828607888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc988100e90 msgr2=0x7fc98819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:19.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.120+0000 7fc985e00640 1 --2- 172.21.15.159:0/2828607888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc988100e90 0x7fc98819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:19.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.120+0000 7fc985e00640 1 -- 172.21.15.159:0/2828607888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc988101860 msgr2=0x7fc9881a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:19.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.120+0000 7fc985e00640 1 --2- 172.21.15.159:0/2828607888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc988101860 0x7fc9881a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:19.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.120+0000 7fc985e00640 1 -- 172.21.15.159:0/2828607888 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9881a1d90 con 0x7fc988100060 2024-09-19T07:33:19.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.120+0000 7fc986800640 1 --2- 172.21.15.159:0/2828607888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc988101860 0x7fc9881a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:19.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.120+0000 7fc985e00640 1 --2- 172.21.15.159:0/2828607888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc988100060 0x7fc98819a160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc978009230 tx=0x7fc978031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:19.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.120+0000 7fc97ea00640 1 -- 172.21.15.159:0/2828607888 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97803f030 con 0x7fc988100060 2024-09-19T07:33:19.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.120+0000 7fc97ea00640 1 -- 172.21.15.159:0/2828607888 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc978003ad0 con 0x7fc988100060 2024-09-19T07:33:19.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.120+0000 7fc97ea00640 1 -- 172.21.15.159:0/2828607888 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978035070 con 0x7fc988100060 2024-09-19T07:33:19.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.120+0000 7fc98ce00640 1 -- 172.21.15.159:0/2828607888 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9881a2020 con 0x7fc988100060 2024-09-19T07:33:19.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.121+0000 7fc98ce00640 1 -- 172.21.15.159:0/2828607888 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9881a2550 con 0x7fc988100060 2024-09-19T07:33:19.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.122+0000 7fc98ce00640 1 -- 172.21.15.159:0/2828607888 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc98810bf50 con 0x7fc988100060 2024-09-19T07:33:19.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.123+0000 7fc97ea00640 1 -- 172.21.15.159:0/2828607888 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc978042030 con 0x7fc988100060 2024-09-19T07:33:19.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.125+0000 7fc97ea00640 1 --2- 172.21.15.159:0/2828607888 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc96c0a3380 0x7fc96c0a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:19.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.125+0000 7fc97ea00640 1 -- 172.21.15.159:0/2828607888 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fc9780effa0 con 0x7fc988100060 2024-09-19T07:33:19.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.125+0000 7fc985400640 1 --2- 172.21.15.159:0/2828607888 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc96c0a3380 0x7fc96c0a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:19.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.125+0000 7fc985400640 1 --2- 172.21.15.159:0/2828607888 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc96c0a3380 0x7fc96c0a5840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc98819b680 tx=0x7fc974005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:19.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.129+0000 7fc97ea00640 1 -- 172.21.15.159:0/2828607888 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc9780bc100 con 0x7fc988100060 2024-09-19T07:33:19.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.318+0000 7fc98ce00640 1 -- 172.21.15.159:0/2828607888 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fc988062630 con 0x7fc96c0a3380 2024-09-19T07:33:19.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.319+0000 7fc97ea00640 1 -- 172.21.15.159:0/2828607888 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fc988062630 con 0x7fc96c0a3380 2024-09-19T07:33:19.320 INFO:teuthology.orchestra.run.smithi159.stdout:Backend: cephadm 2024-09-19T07:33:19.320 INFO:teuthology.orchestra.run.smithi159.stdout:Available: Yes 2024-09-19T07:33:19.320 INFO:teuthology.orchestra.run.smithi159.stdout:Paused: No 2024-09-19T07:33:19.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.322+0000 7fc98ce00640 1 -- 172.21.15.159:0/2828607888 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc96c0a3380 msgr2=0x7fc96c0a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:19.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.322+0000 7fc98ce00640 1 --2- 172.21.15.159:0/2828607888 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc96c0a3380 0x7fc96c0a5840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc98819b680 tx=0x7fc974005f90 comp rx=0 tx=0).stop 2024-09-19T07:33:19.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.322+0000 7fc98ce00640 1 -- 172.21.15.159:0/2828607888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc988100060 msgr2=0x7fc98819a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:19.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.322+0000 7fc98ce00640 1 --2- 172.21.15.159:0/2828607888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc988100060 0x7fc98819a160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc978009230 tx=0x7fc978031b60 comp rx=0 tx=0).stop 2024-09-19T07:33:19.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.322+0000 7fc98ce00640 1 -- 172.21.15.159:0/2828607888 shutdown_connections 2024-09-19T07:33:19.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.322+0000 7fc98ce00640 1 --2- 172.21.15.159:0/2828607888 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc96c0a3380 0x7fc96c0a5840 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:19.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.322+0000 7fc98ce00640 1 --2- 172.21.15.159:0/2828607888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc988101860 0x7fc9881a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:19.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.322+0000 7fc98ce00640 1 --2- 172.21.15.159:0/2828607888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc988100e90 0x7fc98819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:19.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.322+0000 7fc98ce00640 1 --2- 172.21.15.159:0/2828607888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc988100060 0x7fc98819a160 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:19.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.322+0000 7fc98ce00640 1 -- 172.21.15.159:0/2828607888 >> 172.21.15.159:0/2828607888 conn(0x7fc9880fb610 msgr2=0x7fc9880fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:19.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.323+0000 7fc98ce00640 1 -- 172.21.15.159:0/2828607888 shutdown_connections 2024-09-19T07:33:19.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:19.323+0000 7fc98ce00640 1 -- 172.21.15.159:0/2828607888 wait complete. 2024-09-19T07:33:20.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:19 smithi189 ceph-mon[34851]: pgmap v384: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:33:20.077 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:19 smithi159 ceph-mon[30472]: pgmap v384: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:33:20.118 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T07:33:20.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:19 smithi203 ceph-mon[35335]: pgmap v384: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:33:20.489 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:20.889 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:20 smithi159 ceph-mon[30472]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:21.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:20 smithi189 ceph-mon[34851]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:21.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.117+0000 7f43fbe00640 1 -- 172.21.15.159:0/3700859202 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43f4101810 msgr2=0x7f43f40ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:21.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.117+0000 7f43fbe00640 1 --2- 172.21.15.159:0/3700859202 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43f4101810 0x7f43f40ff9a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009900 tx=0x7f43e002a410 comp rx=0 tx=0).stop 2024-09-19T07:33:21.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.118+0000 7f43fbe00640 1 -- 172.21.15.159:0/3700859202 shutdown_connections 2024-09-19T07:33:21.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.118+0000 7f43fbe00640 1 --2- 172.21.15.159:0/3700859202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43f4100090 0x7f43f4101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:21.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.118+0000 7f43fbe00640 1 --2- 172.21.15.159:0/3700859202 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43f4101810 0x7f43f40ff9a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:21.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.118+0000 7f43fbe00640 1 --2- 172.21.15.159:0/3700859202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43f4100e40 0x7f43f4101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:21.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.118+0000 7f43fbe00640 1 -- 172.21.15.159:0/3700859202 >> 172.21.15.159:0/3700859202 conn(0x7f43f40fb610 msgr2=0x7f43f40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:21.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.118+0000 7f43fbe00640 1 -- 172.21.15.159:0/3700859202 shutdown_connections 2024-09-19T07:33:21.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.118+0000 7f43fbe00640 1 -- 172.21.15.159:0/3700859202 wait complete. 2024-09-19T07:33:21.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.119+0000 7f43fbe00640 1 Processor -- start 2024-09-19T07:33:21.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.119+0000 7f43fbe00640 1 -- start start 2024-09-19T07:33:21.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.119+0000 7f43fbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43f4100090 0x7f43f419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:21.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.119+0000 7f43fbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43f4100e40 0x7f43f419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:21.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.119+0000 7f43fbe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43f4101810 0x7f43f41a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:21.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.119+0000 7f43fbe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f43f410fac0 con 0x7f43f4100090 2024-09-19T07:33:21.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.119+0000 7f43fbe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43f410f940 con 0x7f43f4100e40 2024-09-19T07:33:21.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.119+0000 7f43fbe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f43f410fc40 con 0x7f43f4101810 2024-09-19T07:33:21.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.119+0000 7f43f9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43f4100090 0x7f43f419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:21.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.119+0000 7f43f9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43f4100090 0x7f43f419a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:57220/0 (socket says 172.21.15.159:57220) 2024-09-19T07:33:21.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.119+0000 7f43f9400640 1 -- 172.21.15.159:0/3226902175 learned_addr learned my addr 172.21.15.159:0/3226902175 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:21.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.120+0000 7f43f9e00640 1 --2- 172.21.15.159:0/3226902175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43f4101810 0x7f43f41a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:21.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.120+0000 7f43f8a00640 1 --2- 172.21.15.159:0/3226902175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43f4100e40 0x7f43f419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:21.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.120+0000 7f43f9400640 1 -- 172.21.15.159:0/3226902175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43f4101810 msgr2=0x7f43f41a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:21.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.120+0000 7f43f9400640 1 --2- 172.21.15.159:0/3226902175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43f4101810 0x7f43f41a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:21.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.120+0000 7f43f9400640 1 -- 172.21.15.159:0/3226902175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43f4100e40 msgr2=0x7f43f419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:21.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.120+0000 7f43f9400640 1 --2- 172.21.15.159:0/3226902175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43f4100e40 0x7f43f419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:21.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.120+0000 7f43f9400640 1 -- 172.21.15.159:0/3226902175 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f41a1d80 con 0x7f43f4100090 2024-09-19T07:33:21.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.120+0000 7f43f9400640 1 --2- 172.21.15.159:0/3226902175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43f4100090 0x7f43f419a1c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f43e800e9f0 tx=0x7f43e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:21.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.120+0000 7f43f2000640 1 -- 172.21.15.159:0/3226902175 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e800cd80 con 0x7f43f4100090 2024-09-19T07:33:21.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.121+0000 7f43fbe00640 1 -- 172.21.15.159:0/3226902175 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f41a2070 con 0x7f43f4100090 2024-09-19T07:33:21.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.121+0000 7f43f2000640 1 -- 172.21.15.159:0/3226902175 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43e8005500 con 0x7f43f4100090 2024-09-19T07:33:21.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.121+0000 7f43f2000640 1 -- 172.21.15.159:0/3226902175 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8010690 con 0x7f43f4100090 2024-09-19T07:33:21.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.121+0000 7f43fbe00640 1 -- 172.21.15.159:0/3226902175 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f41a2530 con 0x7f43f4100090 2024-09-19T07:33:21.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.122+0000 7f43fbe00640 1 -- 172.21.15.159:0/3226902175 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f410bf30 con 0x7f43f4100090 2024-09-19T07:33:21.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.124+0000 7f43f2000640 1 -- 172.21.15.159:0/3226902175 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f43e80046c0 con 0x7f43f4100090 2024-09-19T07:33:21.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.125+0000 7f43f2000640 1 --2- 172.21.15.159:0/3226902175 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f43dc0a3300 0x7f43dc0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:21.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.125+0000 7f43f2000640 1 -- 172.21.15.159:0/3226902175 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f43e80d0290 con 0x7f43f4100090 2024-09-19T07:33:21.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.125+0000 7f43f8a00640 1 --2- 172.21.15.159:0/3226902175 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f43dc0a3300 0x7f43dc0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:21.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.126+0000 7f43f8a00640 1 --2- 172.21.15.159:0/3226902175 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f43dc0a3300 0x7f43dc0a57c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f43e002aa00 tx=0x7f43e00315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:21.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.129+0000 7f43f2000640 1 -- 172.21.15.159:0/3226902175 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f43e809ca40 con 0x7f43f4100090 2024-09-19T07:33:21.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:20 smithi203 ceph-mon[35335]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:21.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.312+0000 7f43fbe00640 1 -- 172.21.15.159:0/3226902175 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f43f4062630 con 0x7f43dc0a3300 2024-09-19T07:33:21.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.325+0000 7f43f2000640 1 -- 172.21.15.159:0/3226902175 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f43f4062630 con 0x7f43dc0a3300 2024-09-19T07:33:21.327 INFO:teuthology.orchestra.run.smithi159.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T07:33:21.327 INFO:teuthology.orchestra.run.smithi159.stdout:mds.a smithi159 running (2m) 14s ago 2m 13.0M - 19.1.1-361-g2f599a46 d2baa855c067 202929e4eb2d 2024-09-19T07:33:21.327 INFO:teuthology.orchestra.run.smithi159.stdout:mds.b smithi189 running (2m) 17s ago 2m 29.2M - 19.1.1-361-g2f599a46 d2baa855c067 26d2b7f256a5 2024-09-19T07:33:21.327 INFO:teuthology.orchestra.run.smithi159.stdout:mds.c smithi203 running (2m) 17s ago 2m 18.6M - 19.1.1-361-g2f599a46 d2baa855c067 1e80ff8e5053 2024-09-19T07:33:21.327 INFO:teuthology.orchestra.run.smithi159.stdout:mds.d smithi159 running (2m) 14s ago 2m 13.0M - 19.1.1-361-g2f599a46 d2baa855c067 637c6bb5df7e 2024-09-19T07:33:21.327 INFO:teuthology.orchestra.run.smithi159.stdout:mds.e smithi189 running (2m) 17s ago 2m 18.4M - 19.1.1-361-g2f599a46 d2baa855c067 aa4b2d77bfa0 2024-09-19T07:33:21.327 INFO:teuthology.orchestra.run.smithi159.stdout:mds.f smithi203 running (2m) 17s ago 2m 20.4M - 19.1.1-361-g2f599a46 d2baa855c067 fc71e70c06dd 2024-09-19T07:33:21.327 INFO:teuthology.orchestra.run.smithi159.stdout:mds.g smithi159 running (2m) 14s ago 2m 13.0M - 19.1.1-361-g2f599a46 d2baa855c067 9ef3eb9dc557 2024-09-19T07:33:21.327 INFO:teuthology.orchestra.run.smithi159.stdout:mds.h smithi189 running (2m) 17s ago 2m 21.3M - 19.1.1-361-g2f599a46 d2baa855c067 806c9ecaa4a3 2024-09-19T07:33:21.327 INFO:teuthology.orchestra.run.smithi159.stdout:mds.i smithi203 running (2m) 17s ago 2m 20.5M - 19.1.1-361-g2f599a46 d2baa855c067 826b5b583f2d 2024-09-19T07:33:21.328 INFO:teuthology.orchestra.run.smithi159.stdout:mds.j smithi159 running (3m) 14s ago 3m 13.2M - 19.1.1-361-g2f599a46 d2baa855c067 235cea9acf8e 2024-09-19T07:33:21.328 INFO:teuthology.orchestra.run.smithi159.stdout:mds.k smithi189 running (3m) 17s ago 3m 14.9M - 19.1.1-361-g2f599a46 d2baa855c067 d03627f586fa 2024-09-19T07:33:21.328 INFO:teuthology.orchestra.run.smithi159.stdout:mds.l smithi203 running (3m) 17s ago 3m 14.6M - 19.1.1-361-g2f599a46 d2baa855c067 8c453e9aff1e 2024-09-19T07:33:21.328 INFO:teuthology.orchestra.run.smithi159.stdout:mgr.x smithi159 *:9283,8765 running (10m) 14s ago 10m 512M - 19.1.1-361-g2f599a46 d2baa855c067 69b92e6f3945 2024-09-19T07:33:21.328 INFO:teuthology.orchestra.run.smithi159.stdout:mgr.y smithi189 *:8443,8765 running (8m) 17s ago 8m 441M - 19.1.1-361-g2f599a46 d2baa855c067 315155a80885 2024-09-19T07:33:21.328 INFO:teuthology.orchestra.run.smithi159.stdout:mgr.z smithi203 *:8443,8765 running (8m) 17s ago 8m 443M - 19.1.1-361-g2f599a46 d2baa855c067 92614ce544d1 2024-09-19T07:33:21.328 INFO:teuthology.orchestra.run.smithi159.stdout:mon.a smithi159 running (10m) 14s ago 10m 49.8M 2048M 19.1.1-361-g2f599a46 d2baa855c067 37afdc5581e3 2024-09-19T07:33:21.328 INFO:teuthology.orchestra.run.smithi159.stdout:mon.b smithi189 running (9m) 17s ago 9m 44.1M 2048M 19.1.1-361-g2f599a46 d2baa855c067 e3b690676dfc 2024-09-19T07:33:21.328 INFO:teuthology.orchestra.run.smithi159.stdout:mon.c smithi203 running (9m) 17s ago 9m 44.7M 2048M 19.1.1-361-g2f599a46 d2baa855c067 9bc3c2e8fa41 2024-09-19T07:33:21.328 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0 smithi159 running (8m) 14s ago 8m 56.4M 4272M 19.1.1-361-g2f599a46 d2baa855c067 e15ba3bfdc4b 2024-09-19T07:33:21.328 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1 smithi189 running (8m) 17s ago 8m 50.5M 4272M 19.1.1-361-g2f599a46 d2baa855c067 2fafa1e5599f 2024-09-19T07:33:21.328 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2 smithi203 running (7m) 17s ago 7m 47.3M 4272M 19.1.1-361-g2f599a46 d2baa855c067 cf6bc09e2c60 2024-09-19T07:33:21.328 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3 smithi159 running (7m) 14s ago 7m 53.8M 4272M 19.1.1-361-g2f599a46 d2baa855c067 d4826fbbdb5e 2024-09-19T07:33:21.328 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4 smithi189 running (6m) 17s ago 6m 43.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 5507a3924330 2024-09-19T07:33:21.328 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5 smithi203 running (6m) 17s ago 6m 48.3M 4272M 19.1.1-361-g2f599a46 d2baa855c067 03ada340798b 2024-09-19T07:33:21.329 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6 smithi159 running (5m) 14s ago 5m 43.3M 4272M 19.1.1-361-g2f599a46 d2baa855c067 e4d3c78cff89 2024-09-19T07:33:21.329 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7 smithi189 running (5m) 17s ago 5m 47.1M 4272M 19.1.1-361-g2f599a46 d2baa855c067 942984d78e4c 2024-09-19T07:33:21.329 INFO:teuthology.orchestra.run.smithi159.stdout:osd.8 smithi203 running (4m) 17s ago 4m 55.3M 4272M 19.1.1-361-g2f599a46 d2baa855c067 2e6398352d4b 2024-09-19T07:33:21.329 INFO:teuthology.orchestra.run.smithi159.stdout:osd.9 smithi159 running (4m) 14s ago 4m 42.3M 4272M 19.1.1-361-g2f599a46 d2baa855c067 0f7d772f1954 2024-09-19T07:33:21.329 INFO:teuthology.orchestra.run.smithi159.stdout:osd.10 smithi189 running (3m) 17s ago 3m 45.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 ee33c5f623ea 2024-09-19T07:33:21.329 INFO:teuthology.orchestra.run.smithi159.stdout:osd.11 smithi203 running (3m) 17s ago 3m 52.1M 4272M 19.1.1-361-g2f599a46 d2baa855c067 9d16939d78d3 2024-09-19T07:33:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.329+0000 7f43fbe00640 1 -- 172.21.15.159:0/3226902175 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f43dc0a3300 msgr2=0x7f43dc0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.329+0000 7f43fbe00640 1 --2- 172.21.15.159:0/3226902175 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f43dc0a3300 0x7f43dc0a57c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f43e002aa00 tx=0x7f43e00315b0 comp rx=0 tx=0).stop 2024-09-19T07:33:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.330+0000 7f43fbe00640 1 -- 172.21.15.159:0/3226902175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43f4100090 msgr2=0x7f43f419a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.330+0000 7f43fbe00640 1 --2- 172.21.15.159:0/3226902175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43f4100090 0x7f43f419a1c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f43e800e9f0 tx=0x7f43e800edf0 comp rx=0 tx=0).stop 2024-09-19T07:33:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.330+0000 7f43fbe00640 1 -- 172.21.15.159:0/3226902175 shutdown_connections 2024-09-19T07:33:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.330+0000 7f43fbe00640 1 --2- 172.21.15.159:0/3226902175 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f43dc0a3300 0x7f43dc0a57c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.330+0000 7f43fbe00640 1 --2- 172.21.15.159:0/3226902175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43f4101810 0x7f43f41a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.330+0000 7f43fbe00640 1 --2- 172.21.15.159:0/3226902175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43f4100e40 0x7f43f419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.330+0000 7f43fbe00640 1 --2- 172.21.15.159:0/3226902175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43f4100090 0x7f43f419a1c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.330+0000 7f43fbe00640 1 -- 172.21.15.159:0/3226902175 >> 172.21.15.159:0/3226902175 conn(0x7f43f40fb610 msgr2=0x7f43f40fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.330+0000 7f43fbe00640 1 -- 172.21.15.159:0/3226902175 shutdown_connections 2024-09-19T07:33:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:21.330+0000 7f43fbe00640 1 -- 172.21.15.159:0/3226902175 wait complete. 2024-09-19T07:33:21.992 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:21 smithi159 ceph-mon[30472]: pgmap v385: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:22.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:21 smithi189 ceph-mon[34851]: pgmap v385: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:22.153 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T07:33:22.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:21 smithi203 ceph-mon[35335]: pgmap v385: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:22.546 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:22.951 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:22 smithi159 ceph-mon[30472]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:23.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.229+0000 7f8c02e00640 1 -- 172.21.15.159:0/511560458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bfc104f60 msgr2=0x7f8bfc1053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:23.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.229+0000 7f8c02e00640 1 --2- 172.21.15.159:0/511560458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bfc104f60 0x7f8bfc1053e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009900 tx=0x7f8be802a410 comp rx=0 tx=0).stop 2024-09-19T07:33:23.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.229+0000 7f8c02e00640 1 -- 172.21.15.159:0/511560458 shutdown_connections 2024-09-19T07:33:23.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.229+0000 7f8c02e00640 1 --2- 172.21.15.159:0/511560458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bfc105920 0x7f8bfc10c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:23.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.229+0000 7f8c02e00640 1 --2- 172.21.15.159:0/511560458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bfc104f60 0x7f8bfc1053e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:23.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.229+0000 7f8c02e00640 1 --2- 172.21.15.159:0/511560458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bfc103e00 0x7f8bfc104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:23.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.229+0000 7f8c02e00640 1 -- 172.21.15.159:0/511560458 >> 172.21.15.159:0/511560458 conn(0x7f8bfc0ff5b0 msgr2=0x7f8bfc1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:23.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.230+0000 7f8c02e00640 1 -- 172.21.15.159:0/511560458 shutdown_connections 2024-09-19T07:33:23.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.230+0000 7f8c02e00640 1 -- 172.21.15.159:0/511560458 wait complete. 2024-09-19T07:33:23.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.230+0000 7f8c02e00640 1 Processor -- start 2024-09-19T07:33:23.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.231+0000 7f8c02e00640 1 -- start start 2024-09-19T07:33:23.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.231+0000 7f8c02e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bfc103e00 0x7f8bfc19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:23.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.231+0000 7f8c02e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bfc104f60 0x7f8bfc19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:23.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.231+0000 7f8c02e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bfc105920 0x7f8bfc1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:23.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.231+0000 7f8c02e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc10fb00 con 0x7f8bfc105920 2024-09-19T07:33:23.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.231+0000 7f8c02e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc10f980 con 0x7f8bfc104f60 2024-09-19T07:33:23.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.231+0000 7f8c02e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc10fc80 con 0x7f8bfc103e00 2024-09-19T07:33:23.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.231+0000 7f8c00e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bfc105920 0x7f8bfc1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:23.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.231+0000 7f8c00e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bfc105920 0x7f8bfc1a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:46986/0 (socket says 172.21.15.159:46986) 2024-09-19T07:33:23.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.231+0000 7f8c00e00640 1 -- 172.21.15.159:0/286207051 learned_addr learned my addr 172.21.15.159:0/286207051 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:23.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.231+0000 7f8bfbe00640 1 --2- 172.21.15.159:0/286207051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bfc103e00 0x7f8bfc19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:23.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.231+0000 7f8bfb400640 1 --2- 172.21.15.159:0/286207051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bfc104f60 0x7f8bfc19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:23.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.232+0000 7f8c00e00640 1 -- 172.21.15.159:0/286207051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bfc103e00 msgr2=0x7f8bfc19a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:23.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.232+0000 7f8c00e00640 1 --2- 172.21.15.159:0/286207051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bfc103e00 0x7f8bfc19a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:23.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.232+0000 7f8c00e00640 1 -- 172.21.15.159:0/286207051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bfc104f60 msgr2=0x7f8bfc19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:23.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.232+0000 7f8c00e00640 1 --2- 172.21.15.159:0/286207051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bfc104f60 0x7f8bfc19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:23.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.232+0000 7f8c00e00640 1 -- 172.21.15.159:0/286207051 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bfc1a1d60 con 0x7f8bfc105920 2024-09-19T07:33:23.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.232+0000 7f8bfbe00640 1 --2- 172.21.15.159:0/286207051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bfc103e00 0x7f8bfc19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:23.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.232+0000 7f8bfb400640 1 --2- 172.21.15.159:0/286207051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bfc104f60 0x7f8bfc19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:23.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.232+0000 7f8c00e00640 1 --2- 172.21.15.159:0/286207051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bfc105920 0x7f8bfc1a16f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400eb10 tx=0x7f8bf400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:23.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.232+0000 7f8bf8c00640 1 -- 172.21.15.159:0/286207051 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4004b20 con 0x7f8bfc105920 2024-09-19T07:33:23.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.232+0000 7f8bf8c00640 1 -- 172.21.15.159:0/286207051 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bf4004cc0 con 0x7f8bfc105920 2024-09-19T07:33:23.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.232+0000 7f8c02e00640 1 -- 172.21.15.159:0/286207051 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bfc1a1ff0 con 0x7f8bfc105920 2024-09-19T07:33:23.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.232+0000 7f8bf8c00640 1 -- 172.21.15.159:0/286207051 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4010760 con 0x7f8bfc105920 2024-09-19T07:33:23.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.232+0000 7f8c02e00640 1 -- 172.21.15.159:0/286207051 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bfc1a2500 con 0x7f8bfc105920 2024-09-19T07:33:23.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.234+0000 7f8c02e00640 1 -- 172.21.15.159:0/286207051 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bfc10b240 con 0x7f8bfc105920 2024-09-19T07:33:23.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.236+0000 7f8bf8c00640 1 -- 172.21.15.159:0/286207051 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8bf4005020 con 0x7f8bfc105920 2024-09-19T07:33:23.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.237+0000 7f8bf8c00640 1 --2- 172.21.15.159:0/286207051 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8be00a3380 0x7f8be00a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:23.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.237+0000 7f8bf8c00640 1 -- 172.21.15.159:0/286207051 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f8bf40d0390 con 0x7f8bfc105920 2024-09-19T07:33:23.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.238+0000 7f8bfbe00640 1 --2- 172.21.15.159:0/286207051 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8be00a3380 0x7f8be00a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:23.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.238+0000 7f8bfbe00640 1 --2- 172.21.15.159:0/286207051 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8be00a3380 0x7f8be00a5840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009fc0 tx=0x7f8bec005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:23.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.241+0000 7f8bf8c00640 1 -- 172.21.15.159:0/286207051 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8bf409cb40 con 0x7f8bfc105920 2024-09-19T07:33:23.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:22 smithi203 ceph-mon[35335]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:23.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:22 smithi189 ceph-mon[34851]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:23.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.425+0000 7f8c02e00640 1 -- 172.21.15.159:0/286207051 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f8bfc062630 con 0x7f8be00a3380 2024-09-19T07:33:23.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.430+0000 7f8bf8c00640 1 -- 172.21.15.159:0/286207051 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f8bfc062630 con 0x7f8be00a3380 2024-09-19T07:33:23.430 INFO:teuthology.orchestra.run.smithi159.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T07:33:23.431 INFO:teuthology.orchestra.run.smithi159.stdout:mds.all 12/12 19s ago 3m smithi159=a;smithi159=d;smithi159=g;smithi159=j;smithi189=b;smithi189=e;smithi189=h;smithi189=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-09-19T07:33:23.431 INFO:teuthology.orchestra.run.smithi159.stdout:mgr 3/3 19s ago 8m smithi159=x;smithi189=y;smithi203=z;count:3 2024-09-19T07:33:23.431 INFO:teuthology.orchestra.run.smithi159.stdout:mon 3/3 19s ago 9m smithi159:172.21.15.159=a;smithi189:172.21.15.189=b;smithi203:172.21.15.203=c;count:3 2024-09-19T07:33:23.431 INFO:teuthology.orchestra.run.smithi159.stdout:osd 12 19s ago - 2024-09-19T07:33:23.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.433+0000 7f8c02e00640 1 -- 172.21.15.159:0/286207051 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8be00a3380 msgr2=0x7f8be00a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:23.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.433+0000 7f8c02e00640 1 --2- 172.21.15.159:0/286207051 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8be00a3380 0x7f8be00a5840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009fc0 tx=0x7f8bec005f90 comp rx=0 tx=0).stop 2024-09-19T07:33:23.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.433+0000 7f8c02e00640 1 -- 172.21.15.159:0/286207051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bfc105920 msgr2=0x7f8bfc1a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:23.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.433+0000 7f8c02e00640 1 --2- 172.21.15.159:0/286207051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bfc105920 0x7f8bfc1a16f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400eb10 tx=0x7f8bf400c2d0 comp rx=0 tx=0).stop 2024-09-19T07:33:23.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.433+0000 7f8c02e00640 1 -- 172.21.15.159:0/286207051 shutdown_connections 2024-09-19T07:33:23.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.433+0000 7f8c02e00640 1 --2- 172.21.15.159:0/286207051 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f8be00a3380 0x7f8be00a5840 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:23.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.433+0000 7f8c02e00640 1 --2- 172.21.15.159:0/286207051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bfc105920 0x7f8bfc1a16f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:23.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.433+0000 7f8c02e00640 1 --2- 172.21.15.159:0/286207051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bfc104f60 0x7f8bfc19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:23.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.433+0000 7f8c02e00640 1 --2- 172.21.15.159:0/286207051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bfc103e00 0x7f8bfc19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:23.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.433+0000 7f8c02e00640 1 -- 172.21.15.159:0/286207051 >> 172.21.15.159:0/286207051 conn(0x7f8bfc0ff5b0 msgr2=0x7f8bfc108410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:23.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.433+0000 7f8c02e00640 1 -- 172.21.15.159:0/286207051 shutdown_connections 2024-09-19T07:33:23.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:23.433+0000 7f8c02e00640 1 -- 172.21.15.159:0/286207051 wait complete. 2024-09-19T07:33:24.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:23 smithi159 ceph-mon[30472]: pgmap v386: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:33:24.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:23 smithi203 ceph-mon[35335]: pgmap v386: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:33:24.294 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T07:33:24.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:23 smithi189 ceph-mon[34851]: pgmap v386: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:33:24.664 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:25.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:24 smithi159 ceph-mon[30472]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:25.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.279+0000 7f3ca4600640 1 -- 172.21.15.159:0/4270167334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c9c102e00 msgr2=0x7f3c9c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:25.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.279+0000 7f3ca4600640 1 --2- 172.21.15.159:0/4270167334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c9c102e00 0x7f3c9c103280 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3c88009900 tx=0x7f3c8802a410 comp rx=0 tx=0).stop 2024-09-19T07:33:25.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.279+0000 7f3ca4600640 1 -- 172.21.15.159:0/4270167334 shutdown_connections 2024-09-19T07:33:25.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.279+0000 7f3ca4600640 1 --2- 172.21.15.159:0/4270167334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c9c1037c0 0x7f3c9c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:25.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.279+0000 7f3ca4600640 1 --2- 172.21.15.159:0/4270167334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c9c102e00 0x7f3c9c103280 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:25.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.279+0000 7f3ca4600640 1 --2- 172.21.15.159:0/4270167334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c9c101c00 0x7f3c9c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:25.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.279+0000 7f3ca4600640 1 -- 172.21.15.159:0/4270167334 >> 172.21.15.159:0/4270167334 conn(0x7f3c9c0fd390 msgr2=0x7f3c9c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:25.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.280+0000 7f3ca4600640 1 -- 172.21.15.159:0/4270167334 shutdown_connections 2024-09-19T07:33:25.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.280+0000 7f3ca4600640 1 -- 172.21.15.159:0/4270167334 wait complete. 2024-09-19T07:33:25.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.280+0000 7f3ca4600640 1 Processor -- start 2024-09-19T07:33:25.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.281+0000 7f3ca4600640 1 -- start start 2024-09-19T07:33:25.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.281+0000 7f3ca4600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c9c101c00 0x7f3c9c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:25.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.281+0000 7f3ca4600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c9c102e00 0x7f3c9c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:25.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.281+0000 7f3ca4600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c9c1037c0 0x7f3c9c19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:25.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.281+0000 7f3ca4600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c10d880 con 0x7f3c9c1037c0 2024-09-19T07:33:25.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.281+0000 7f3ca4600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c10d700 con 0x7f3c9c101c00 2024-09-19T07:33:25.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.281+0000 7f3ca4600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c10da00 con 0x7f3c9c102e00 2024-09-19T07:33:25.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.281+0000 7f3ca2600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c9c1037c0 0x7f3c9c19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:25.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.281+0000 7f3ca1200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c9c102e00 0x7f3c9c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:25.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.281+0000 7f3ca1200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c9c102e00 0x7f3c9c198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:60848/0 (socket says 172.21.15.159:60848) 2024-09-19T07:33:25.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.281+0000 7f3ca1c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c9c101c00 0x7f3c9c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:25.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.281+0000 7f3ca1200640 1 -- 172.21.15.159:0/1980232300 learned_addr learned my addr 172.21.15.159:0/1980232300 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:25.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.282+0000 7f3ca1c00640 1 -- 172.21.15.159:0/1980232300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c9c102e00 msgr2=0x7f3c9c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:25.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.282+0000 7f3ca1c00640 1 --2- 172.21.15.159:0/1980232300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c9c102e00 0x7f3c9c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:25.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.282+0000 7f3ca1c00640 1 -- 172.21.15.159:0/1980232300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c9c1037c0 msgr2=0x7f3c9c19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:25.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.282+0000 7f3ca1c00640 1 --2- 172.21.15.159:0/1980232300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c9c1037c0 0x7f3c9c19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:25.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.282+0000 7f3ca1c00640 1 -- 172.21.15.159:0/1980232300 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9c19faf0 con 0x7f3c9c101c00 2024-09-19T07:33:25.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.282+0000 7f3ca1c00640 1 --2- 172.21.15.159:0/1980232300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c9c101c00 0x7f3c9c197ec0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c102c60 tx=0x7f3c9000dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:25.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.282+0000 7f3c9a000640 1 -- 172.21.15.159:0/1980232300 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c90017070 con 0x7f3c9c101c00 2024-09-19T07:33:25.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.282+0000 7f3ca4600640 1 -- 172.21.15.159:0/1980232300 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9c19fde0 con 0x7f3c9c101c00 2024-09-19T07:33:25.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.282+0000 7f3c9a000640 1 -- 172.21.15.159:0/1980232300 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c90009c30 con 0x7f3c9c101c00 2024-09-19T07:33:25.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.282+0000 7f3c9a000640 1 -- 172.21.15.159:0/1980232300 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9000f620 con 0x7f3c9c101c00 2024-09-19T07:33:25.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.283+0000 7f3ca4600640 1 -- 172.21.15.159:0/1980232300 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c9c1a0240 con 0x7f3c9c101c00 2024-09-19T07:33:25.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:24 smithi203 ceph-mon[35335]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:25.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.284+0000 7f3ca4600640 1 -- 172.21.15.159:0/1980232300 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c9c108fa0 con 0x7f3c9c101c00 2024-09-19T07:33:25.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.286+0000 7f3c9a000640 1 -- 172.21.15.159:0/1980232300 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3c9001a070 con 0x7f3c9c101c00 2024-09-19T07:33:25.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.287+0000 7f3c9a000640 1 --2- 172.21.15.159:0/1980232300 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3c840a3380 0x7f3c840a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:25.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.287+0000 7f3c9a000640 1 -- 172.21.15.159:0/1980232300 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3c900cf780 con 0x7f3c9c101c00 2024-09-19T07:33:25.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.287+0000 7f3ca1200640 1 --2- 172.21.15.159:0/1980232300 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3c840a3380 0x7f3c840a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:25.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.288+0000 7f3ca1200640 1 --2- 172.21.15.159:0/1980232300 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3c840a3380 0x7f3c840a5840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3c88031440 tx=0x7f3c880315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:25.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.291+0000 7f3c9a000640 1 -- 172.21.15.159:0/1980232300 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3c9009b8e0 con 0x7f3c9c101c00 2024-09-19T07:33:25.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:24 smithi189 ceph-mon[34851]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:25.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.471+0000 7f3ca4600640 1 -- 172.21.15.159:0/1980232300 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f3c9c062630 con 0x7f3c840a3380 2024-09-19T07:33:25.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.473+0000 7f3c9a000640 1 -- 172.21.15.159:0/1980232300 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f3c9c062630 con 0x7f3c840a3380 2024-09-19T07:33:25.473 INFO:teuthology.orchestra.run.smithi159.stdout:HOST ADDR LABELS STATUS 2024-09-19T07:33:25.473 INFO:teuthology.orchestra.run.smithi159.stdout:smithi159 172.21.15.159 2024-09-19T07:33:25.473 INFO:teuthology.orchestra.run.smithi159.stdout:smithi189 172.21.15.189 2024-09-19T07:33:25.473 INFO:teuthology.orchestra.run.smithi159.stdout:smithi203 172.21.15.203 2024-09-19T07:33:25.473 INFO:teuthology.orchestra.run.smithi159.stdout:3 hosts in cluster 2024-09-19T07:33:25.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.475+0000 7f3ca4600640 1 -- 172.21.15.159:0/1980232300 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3c840a3380 msgr2=0x7f3c840a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:25.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.475+0000 7f3ca4600640 1 --2- 172.21.15.159:0/1980232300 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3c840a3380 0x7f3c840a5840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3c88031440 tx=0x7f3c880315b0 comp rx=0 tx=0).stop 2024-09-19T07:33:25.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.475+0000 7f3ca4600640 1 -- 172.21.15.159:0/1980232300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c9c101c00 msgr2=0x7f3c9c197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:25.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.475+0000 7f3ca4600640 1 --2- 172.21.15.159:0/1980232300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c9c101c00 0x7f3c9c197ec0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c102c60 tx=0x7f3c9000dde0 comp rx=0 tx=0).stop 2024-09-19T07:33:25.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.475+0000 7f3ca4600640 1 -- 172.21.15.159:0/1980232300 shutdown_connections 2024-09-19T07:33:25.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.475+0000 7f3ca4600640 1 --2- 172.21.15.159:0/1980232300 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3c840a3380 0x7f3c840a5840 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:25.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.475+0000 7f3ca4600640 1 --2- 172.21.15.159:0/1980232300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c9c1037c0 0x7f3c9c19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:25.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.475+0000 7f3ca4600640 1 --2- 172.21.15.159:0/1980232300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c9c102e00 0x7f3c9c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:25.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.475+0000 7f3ca4600640 1 --2- 172.21.15.159:0/1980232300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c9c101c00 0x7f3c9c197ec0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:25.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.475+0000 7f3ca4600640 1 -- 172.21.15.159:0/1980232300 >> 172.21.15.159:0/1980232300 conn(0x7f3c9c0fd390 msgr2=0x7f3c9c106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:25.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.475+0000 7f3ca4600640 1 -- 172.21.15.159:0/1980232300 shutdown_connections 2024-09-19T07:33:25.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:25.475+0000 7f3ca4600640 1 -- 172.21.15.159:0/1980232300 wait complete. 2024-09-19T07:33:26.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:25 smithi159 ceph-mon[30472]: pgmap v387: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:26.206 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:25 smithi189 ceph-mon[34851]: pgmap v387: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:26.268 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T07:33:26.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:25 smithi203 ceph-mon[35335]: pgmap v387: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:26.641 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:27.099 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:26 smithi159 ceph-mon[30472]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:27.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.212+0000 7fa4dc200640 1 -- 172.21.15.159:0/6130380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4d4102dc0 msgr2=0x7fa4d4105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:27.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.212+0000 7fa4dc200640 1 --2- 172.21.15.159:0/6130380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4d4102dc0 0x7fa4d4105230 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0009900 tx=0x7fa4c002a3f0 comp rx=0 tx=0).stop 2024-09-19T07:33:27.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.212+0000 7fa4dc200640 1 -- 172.21.15.159:0/6130380 shutdown_connections 2024-09-19T07:33:27.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.212+0000 7fa4dc200640 1 --2- 172.21.15.159:0/6130380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4d4105770 0x7fa4d4105c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:27.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.212+0000 7fa4dc200640 1 --2- 172.21.15.159:0/6130380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4d4102dc0 0x7fa4d4105230 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:27.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.212+0000 7fa4dc200640 1 --2- 172.21.15.159:0/6130380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4d4100470 0x7fa4d4102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:27.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.212+0000 7fa4dc200640 1 -- 172.21.15.159:0/6130380 >> 172.21.15.159:0/6130380 conn(0x7fa4d40fc280 msgr2=0x7fa4d40fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:27.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.212+0000 7fa4dc200640 1 -- 172.21.15.159:0/6130380 shutdown_connections 2024-09-19T07:33:27.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.213+0000 7fa4dc200640 1 -- 172.21.15.159:0/6130380 wait complete. 2024-09-19T07:33:27.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.213+0000 7fa4dc200640 1 Processor -- start 2024-09-19T07:33:27.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.214+0000 7fa4dc200640 1 -- start start 2024-09-19T07:33:27.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.214+0000 7fa4dc200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4d4100470 0x7fa4d419e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:27.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.214+0000 7fa4dc200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4d4102dc0 0x7fa4d419ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:27.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.214+0000 7fa4dc200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4d4105770 0x7fa4d41a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:27.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.214+0000 7fa4dc200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d4113e80 con 0x7fa4d4102dc0 2024-09-19T07:33:27.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.214+0000 7fa4dc200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d4113d00 con 0x7fa4d4100470 2024-09-19T07:33:27.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.214+0000 7fa4dc200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d4114000 con 0x7fa4d4105770 2024-09-19T07:33:27.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.214+0000 7fa4d8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4d4102dc0 0x7fa4d419ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:27.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.214+0000 7fa4d9800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4d4100470 0x7fa4d419e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:27.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.214+0000 7fa4d9800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4d4100470 0x7fa4d419e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:43596/0 (socket says 172.21.15.159:43596) 2024-09-19T07:33:27.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.214+0000 7fa4d9800640 1 -- 172.21.15.159:0/4149386814 learned_addr learned my addr 172.21.15.159:0/4149386814 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:27.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.214+0000 7fa4da200640 1 --2- 172.21.15.159:0/4149386814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4d4105770 0x7fa4d41a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:27.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.215+0000 7fa4d9800640 1 -- 172.21.15.159:0/4149386814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4d4105770 msgr2=0x7fa4d41a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:27.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.215+0000 7fa4d9800640 1 --2- 172.21.15.159:0/4149386814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4d4105770 0x7fa4d41a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:27.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.215+0000 7fa4d9800640 1 -- 172.21.15.159:0/4149386814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4d4102dc0 msgr2=0x7fa4d419ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:27.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.215+0000 7fa4d9800640 1 --2- 172.21.15.159:0/4149386814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4d4102dc0 0x7fa4d419ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:27.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.215+0000 7fa4d9800640 1 -- 172.21.15.159:0/4149386814 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d41a6120 con 0x7fa4d4100470 2024-09-19T07:33:27.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.215+0000 7fa4d9800640 1 --2- 172.21.15.159:0/4149386814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4d4100470 0x7fa4d419e4f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800e9f0 tx=0x7fa4c800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:27.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.215+0000 7fa4d2000640 1 -- 172.21.15.159:0/4149386814 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c800cd80 con 0x7fa4d4100470 2024-09-19T07:33:27.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.215+0000 7fa4d2000640 1 -- 172.21.15.159:0/4149386814 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c8005500 con 0x7fa4d4100470 2024-09-19T07:33:27.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.215+0000 7fa4d2000640 1 -- 172.21.15.159:0/4149386814 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8010690 con 0x7fa4d4100470 2024-09-19T07:33:27.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.215+0000 7fa4dc200640 1 -- 172.21.15.159:0/4149386814 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d41a6410 con 0x7fa4d4100470 2024-09-19T07:33:27.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.216+0000 7fa4dc200640 1 -- 172.21.15.159:0/4149386814 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d41a68d0 con 0x7fa4d4100470 2024-09-19T07:33:27.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.217+0000 7fa4dc200640 1 -- 172.21.15.159:0/4149386814 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d40fdd70 con 0x7fa4d4100470 2024-09-19T07:33:27.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.219+0000 7fa4d2000640 1 -- 172.21.15.159:0/4149386814 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fa4c80046c0 con 0x7fa4d4100470 2024-09-19T07:33:27.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.220+0000 7fa4d2000640 1 --2- 172.21.15.159:0/4149386814 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa4bc0a3380 0x7fa4bc0a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:27.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.220+0000 7fa4d2000640 1 -- 172.21.15.159:0/4149386814 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa4c80d0440 con 0x7fa4d4100470 2024-09-19T07:33:27.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.220+0000 7fa4d8e00640 1 --2- 172.21.15.159:0/4149386814 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa4bc0a3380 0x7fa4bc0a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:27.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.221+0000 7fa4d8e00640 1 --2- 172.21.15.159:0/4149386814 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa4bc0a3380 0x7fa4bc0a5840 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa4c00314e0 tx=0x7fa4c0000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:27.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.222+0000 7fa4d2000640 1 -- 172.21.15.159:0/4149386814 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa4c8054210 con 0x7fa4d4100470 2024-09-19T07:33:27.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:26 smithi203 ceph-mon[35335]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:27.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:26 smithi189 ceph-mon[34851]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:27.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.406+0000 7fa4dc200640 1 -- 172.21.15.159:0/4149386814 --> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fa4d4062630 con 0x7fa4bc0a3380 2024-09-19T07:33:27.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.408+0000 7fa4d2000640 1 -- 172.21.15.159:0/4149386814 <== mgr.14148 v2:172.21.15.159:6800/4243905339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fa4d4062630 con 0x7fa4bc0a3380 2024-09-19T07:33:27.408 INFO:teuthology.orchestra.run.smithi159.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T07:33:27.408 INFO:teuthology.orchestra.run.smithi159.stdout:smithi159 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GH400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-09-19T07:33:27.408 INFO:teuthology.orchestra.run.smithi159.stdout:smithi189 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232004L400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-19T07:33:27.408 INFO:teuthology.orchestra.run.smithi159.stdout:smithi203 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-19T07:33:27.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.410+0000 7fa4dc200640 1 -- 172.21.15.159:0/4149386814 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa4bc0a3380 msgr2=0x7fa4bc0a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:27.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.410+0000 7fa4dc200640 1 --2- 172.21.15.159:0/4149386814 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa4bc0a3380 0x7fa4bc0a5840 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa4c00314e0 tx=0x7fa4c0000bc0 comp rx=0 tx=0).stop 2024-09-19T07:33:27.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.410+0000 7fa4dc200640 1 -- 172.21.15.159:0/4149386814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4d4100470 msgr2=0x7fa4d419e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:27.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.410+0000 7fa4dc200640 1 --2- 172.21.15.159:0/4149386814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4d4100470 0x7fa4d419e4f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800e9f0 tx=0x7fa4c800edf0 comp rx=0 tx=0).stop 2024-09-19T07:33:27.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.410+0000 7fa4dc200640 1 -- 172.21.15.159:0/4149386814 shutdown_connections 2024-09-19T07:33:27.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.410+0000 7fa4dc200640 1 --2- 172.21.15.159:0/4149386814 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa4bc0a3380 0x7fa4bc0a5840 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:27.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.410+0000 7fa4dc200640 1 --2- 172.21.15.159:0/4149386814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4d4105770 0x7fa4d41a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:27.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.410+0000 7fa4dc200640 1 --2- 172.21.15.159:0/4149386814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4d4102dc0 0x7fa4d419ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:27.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.410+0000 7fa4dc200640 1 --2- 172.21.15.159:0/4149386814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4d4100470 0x7fa4d419e4f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:27.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.410+0000 7fa4dc200640 1 -- 172.21.15.159:0/4149386814 >> 172.21.15.159:0/4149386814 conn(0x7fa4d40fc280 msgr2=0x7fa4d4100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:27.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.411+0000 7fa4dc200640 1 -- 172.21.15.159:0/4149386814 shutdown_connections 2024-09-19T07:33:27.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:27.411+0000 7fa4dc200640 1 -- 172.21.15.159:0/4149386814 wait complete. 2024-09-19T07:33:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:27 smithi159 ceph-mon[30472]: pgmap v388: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:33:28.226 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T07:33:28.235 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi159.front.sepia.ceph.com 2024-09-19T07:33:28.235 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-19T07:33:28.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:27 smithi203 ceph-mon[35335]: pgmap v388: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:33:28.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:27 smithi189 ceph-mon[34851]: pgmap v388: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:33:28.610 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:29.109 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:28 smithi159 ceph-mon[30472]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:29.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.221+0000 7f96cf000640 1 -- 172.21.15.159:0/1030684096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c8102e00 msgr2=0x7f96c8103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:29.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.221+0000 7f96cf000640 1 --2- 172.21.15.159:0/1030684096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c8102e00 0x7f96c8103280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009860 tx=0x7f96b402a340 comp rx=0 tx=0).stop 2024-09-19T07:33:29.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.221+0000 7f96cf000640 1 -- 172.21.15.159:0/1030684096 shutdown_connections 2024-09-19T07:33:29.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.221+0000 7f96cf000640 1 --2- 172.21.15.159:0/1030684096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96c81037c0 0x7f96c810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:29.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.221+0000 7f96cf000640 1 --2- 172.21.15.159:0/1030684096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c8102e00 0x7f96c8103280 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:29.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.221+0000 7f96cf000640 1 --2- 172.21.15.159:0/1030684096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c8101c00 0x7f96c8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:29.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.221+0000 7f96cf000640 1 -- 172.21.15.159:0/1030684096 >> 172.21.15.159:0/1030684096 conn(0x7f96c80fd390 msgr2=0x7f96c80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:29.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.221+0000 7f96cf000640 1 -- 172.21.15.159:0/1030684096 shutdown_connections 2024-09-19T07:33:29.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.221+0000 7f96cf000640 1 -- 172.21.15.159:0/1030684096 wait complete. 2024-09-19T07:33:29.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.222+0000 7f96cf000640 1 Processor -- start 2024-09-19T07:33:29.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.222+0000 7f96cf000640 1 -- start start 2024-09-19T07:33:29.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.222+0000 7f96cf000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c8101c00 0x7f96c819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:29.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.222+0000 7f96cf000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96c8102e00 0x7f96c819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:29.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.222+0000 7f96cf000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c81037c0 0x7f96c81a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:29.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.222+0000 7f96cf000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f96c810d880 con 0x7f96c8102e00 2024-09-19T07:33:29.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.222+0000 7f96cf000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96c810d700 con 0x7f96c8101c00 2024-09-19T07:33:29.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.222+0000 7f96cf000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f96c810da00 con 0x7f96c81037c0 2024-09-19T07:33:29.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.223+0000 7f96c7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96c8102e00 0x7f96c819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:29.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.223+0000 7f96cd000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c81037c0 0x7f96c81a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:29.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.223+0000 7f96c7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c8101c00 0x7f96c819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:29.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.223+0000 7f96c7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96c8102e00 0x7f96c819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47016/0 (socket says 172.21.15.159:47016) 2024-09-19T07:33:29.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.223+0000 7f96c7400640 1 -- 172.21.15.159:0/274272358 learned_addr learned my addr 172.21.15.159:0/274272358 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:29.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.223+0000 7f96cd000640 1 -- 172.21.15.159:0/274272358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c8101c00 msgr2=0x7f96c819a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:29.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.223+0000 7f96cd000640 1 --2- 172.21.15.159:0/274272358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c8101c00 0x7f96c819a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:29.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.223+0000 7f96cd000640 1 -- 172.21.15.159:0/274272358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96c8102e00 msgr2=0x7f96c819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:29.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.223+0000 7f96cd000640 1 --2- 172.21.15.159:0/274272358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96c8102e00 0x7f96c819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:29.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.223+0000 7f96cd000640 1 -- 172.21.15.159:0/274272358 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c81a1d40 con 0x7f96c81037c0 2024-09-19T07:33:29.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.224+0000 7f96cd000640 1 --2- 172.21.15.159:0/274272358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c81037c0 0x7f96c81a16d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f96c000efc0 tx=0x7f96c000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:29.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.224+0000 7f96c4c00640 1 -- 172.21.15.159:0/274272358 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c0003970 con 0x7f96c81037c0 2024-09-19T07:33:29.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.224+0000 7f96c4c00640 1 -- 172.21.15.159:0/274272358 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96c0004020 con 0x7f96c81037c0 2024-09-19T07:33:29.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.224+0000 7f96c4c00640 1 -- 172.21.15.159:0/274272358 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c000b9f0 con 0x7f96c81037c0 2024-09-19T07:33:29.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.224+0000 7f96cf000640 1 -- 172.21.15.159:0/274272358 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c81a1fd0 con 0x7f96c81037c0 2024-09-19T07:33:29.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.224+0000 7f96cf000640 1 -- 172.21.15.159:0/274272358 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c81a2490 con 0x7f96c81037c0 2024-09-19T07:33:29.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.225+0000 7f96cf000640 1 -- 172.21.15.159:0/274272358 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c81a26c0 con 0x7f96c81037c0 2024-09-19T07:33:29.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.227+0000 7f96c4c00640 1 -- 172.21.15.159:0/274272358 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f96c001d020 con 0x7f96c81037c0 2024-09-19T07:33:29.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.228+0000 7f96c4c00640 1 --2- 172.21.15.159:0/274272358 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f96ac0a3300 0x7f96ac0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:29.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.228+0000 7f96c4c00640 1 -- 172.21.15.159:0/274272358 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f96c00cffe0 con 0x7f96c81037c0 2024-09-19T07:33:29.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.229+0000 7f96c7e00640 1 --2- 172.21.15.159:0/274272358 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f96ac0a3300 0x7f96ac0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:29.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.229+0000 7f96c7e00640 1 --2- 172.21.15.159:0/274272358 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f96ac0a3300 0x7f96ac0a57c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f96c8102c60 tx=0x7f96b8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:29.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.231+0000 7f96c4c00640 1 -- 172.21.15.159:0/274272358 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f96c009c7b0 con 0x7f96c81037c0 2024-09-19T07:33:29.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:28 smithi203 ceph-mon[35335]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:29.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:28 smithi189 ceph-mon[34851]: from='client.25046 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:29.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.444+0000 7f96cf000640 1 -- 172.21.15.159:0/274272358 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f96c81a29b0 con 0x7f96c81037c0 2024-09-19T07:33:29.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.444+0000 7f96c4c00640 1 -- 172.21.15.159:0/274272358 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3104 (secure 0 0 0) 0x7f96c009c0b0 con 0x7f96c81037c0 2024-09-19T07:33:29.445 INFO:teuthology.orchestra.run.smithi159.stdout:e25 2024-09-19T07:33:29.445 INFO:teuthology.orchestra.run.smithi159.stdout:btime 2024-09-19T07:31:54:895489+0000 2024-09-19T07:33:29.445 INFO:teuthology.orchestra.run.smithi159.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T07:33:29.446 INFO:teuthology.orchestra.run.smithi159.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T07:33:29.446 INFO:teuthology.orchestra.run.smithi159.stdout:legacy client fscid: 1 2024-09-19T07:33:29.446 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:33:29.446 INFO:teuthology.orchestra.run.smithi159.stdout:Filesystem 'cephfs' (1) 2024-09-19T07:33:29.446 INFO:teuthology.orchestra.run.smithi159.stdout:fs_name cephfs 2024-09-19T07:33:29.446 INFO:teuthology.orchestra.run.smithi159.stdout:epoch 25 2024-09-19T07:33:29.446 INFO:teuthology.orchestra.run.smithi159.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-19T07:33:29.446 INFO:teuthology.orchestra.run.smithi159.stdout:created 2024-09-19T07:31:03.518840+0000 2024-09-19T07:33:29.446 INFO:teuthology.orchestra.run.smithi159.stdout:modified 2024-09-19T07:31:53.954806+0000 2024-09-19T07:33:29.446 INFO:teuthology.orchestra.run.smithi159.stdout:tableserver 0 2024-09-19T07:33:29.446 INFO:teuthology.orchestra.run.smithi159.stdout:root 0 2024-09-19T07:33:29.446 INFO:teuthology.orchestra.run.smithi159.stdout:session_timeout 300 2024-09-19T07:33:29.446 INFO:teuthology.orchestra.run.smithi159.stdout:session_autoclose 300 2024-09-19T07:33:29.447 INFO:teuthology.orchestra.run.smithi159.stdout:max_file_size 1099511627776 2024-09-19T07:33:29.447 INFO:teuthology.orchestra.run.smithi159.stdout:max_xattr_size 65536 2024-09-19T07:33:29.447 INFO:teuthology.orchestra.run.smithi159.stdout:required_client_features {} 2024-09-19T07:33:29.447 INFO:teuthology.orchestra.run.smithi159.stdout:last_failure 0 2024-09-19T07:33:29.447 INFO:teuthology.orchestra.run.smithi159.stdout:last_failure_osd_epoch 0 2024-09-19T07:33:29.447 INFO:teuthology.orchestra.run.smithi159.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T07:33:29.447 INFO:teuthology.orchestra.run.smithi159.stdout:max_mds 3 2024-09-19T07:33:29.447 INFO:teuthology.orchestra.run.smithi159.stdout:in 0,1,2 2024-09-19T07:33:29.447 INFO:teuthology.orchestra.run.smithi159.stdout:up {0=24542,1=24514,2=24506} 2024-09-19T07:33:29.447 INFO:teuthology.orchestra.run.smithi159.stdout:failed 2024-09-19T07:33:29.447 INFO:teuthology.orchestra.run.smithi159.stdout:damaged 2024-09-19T07:33:29.447 INFO:teuthology.orchestra.run.smithi159.stdout:stopped 2024-09-19T07:33:29.447 INFO:teuthology.orchestra.run.smithi159.stdout:data_pools [3,4] 2024-09-19T07:33:29.447 INFO:teuthology.orchestra.run.smithi159.stdout:metadata_pool 2 2024-09-19T07:33:29.448 INFO:teuthology.orchestra.run.smithi159.stdout:inline_data disabled 2024-09-19T07:33:29.448 INFO:teuthology.orchestra.run.smithi159.stdout:balancer 2024-09-19T07:33:29.448 INFO:teuthology.orchestra.run.smithi159.stdout:bal_rank_mask -1 2024-09-19T07:33:29.448 INFO:teuthology.orchestra.run.smithi159.stdout:standby_count_wanted 0 2024-09-19T07:33:29.448 INFO:teuthology.orchestra.run.smithi159.stdout:qdb_cluster leader: 24542 members: 24506,24514,24542 2024-09-19T07:33:29.448 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.b{0:24542} state up:active seq 5 addr [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:33:29.448 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.c{0:24499} state up:standby-replay seq 1 addr [v2:172.21.15.203:6836/1053443810,v1:172.21.15.203:6837/1053443810] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:33:29.448 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.i{1:24514} state up:active seq 19 addr [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:33:29.448 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.f{1:24487} state up:standby-replay seq 1 addr [v2:172.21.15.203:6834/937970878,v1:172.21.15.203:6835/937970878] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:33:29.448 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.h{2:24506} state up:active seq 20 addr [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:33:29.448 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.e{2:24485} state up:standby-replay seq 1 addr [v2:172.21.15.189:6834/1340279549,v1:172.21.15.189:6835/1340279549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:33:29.448 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:33:29.448 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:33:29.449 INFO:teuthology.orchestra.run.smithi159.stdout:Standby daemons: 2024-09-19T07:33:29.449 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:33:29.449 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.j{-1:14433} state up:standby seq 1 addr [v2:172.21.15.159:6834/205226977,v1:172.21.15.159:6835/205226977] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:33:29.449 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.d{-1:14445} state up:standby seq 1 addr [v2:172.21.15.159:6836/3464449771,v1:172.21.15.159:6837/3464449771] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:33:29.449 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.g{-1:14487} state up:standby seq 1 addr [v2:172.21.15.159:6838/2036264856,v1:172.21.15.159:6839/2036264856] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:33:29.449 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.a{-1:14499} state up:standby seq 1 addr [v2:172.21.15.159:6840/2874566405,v1:172.21.15.159:6841/2874566405] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:33:29.449 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.l{-1:24445} state up:standby seq 1 addr [v2:172.21.15.203:6832/1415737112,v1:172.21.15.203:6833/1415737112] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:33:29.449 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.k{-1:24449} state up:standby seq 1 addr [v2:172.21.15.189:6832/2799019079,v1:172.21.15.189:6833/2799019079] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:33:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 25 2024-09-19T07:33:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.447+0000 7f96cf000640 1 -- 172.21.15.159:0/274272358 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f96ac0a3300 msgr2=0x7f96ac0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.447+0000 7f96cf000640 1 --2- 172.21.15.159:0/274272358 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f96ac0a3300 0x7f96ac0a57c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f96c8102c60 tx=0x7f96b8005f90 comp rx=0 tx=0).stop 2024-09-19T07:33:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.447+0000 7f96cf000640 1 -- 172.21.15.159:0/274272358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c81037c0 msgr2=0x7f96c81a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.447+0000 7f96cf000640 1 --2- 172.21.15.159:0/274272358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c81037c0 0x7f96c81a16d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f96c000efc0 tx=0x7f96c000c800 comp rx=0 tx=0).stop 2024-09-19T07:33:29.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.447+0000 7f96cf000640 1 -- 172.21.15.159:0/274272358 shutdown_connections 2024-09-19T07:33:29.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.447+0000 7f96cf000640 1 --2- 172.21.15.159:0/274272358 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f96ac0a3300 0x7f96ac0a57c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:29.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.447+0000 7f96cf000640 1 --2- 172.21.15.159:0/274272358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96c81037c0 0x7f96c81a16d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:29.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.447+0000 7f96cf000640 1 --2- 172.21.15.159:0/274272358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96c8102e00 0x7f96c819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:29.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.447+0000 7f96cf000640 1 --2- 172.21.15.159:0/274272358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96c8101c00 0x7f96c819a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:29.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.447+0000 7f96cf000640 1 -- 172.21.15.159:0/274272358 >> 172.21.15.159:0/274272358 conn(0x7f96c80fd390 msgr2=0x7f96c8074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:29.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.447+0000 7f96cf000640 1 -- 172.21.15.159:0/274272358 shutdown_connections 2024-09-19T07:33:29.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:29.447+0000 7f96cf000640 1 -- 172.21.15.159:0/274272358 wait complete. 2024-09-19T07:33:30.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:29 smithi159 ceph-mon[30472]: pgmap v389: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:33:30.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:29 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/274272358' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T07:33:30.209 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-19T07:33:30.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:29 smithi203 ceph-mon[35335]: pgmap v389: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:33:30.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:29 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/274272358' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T07:33:30.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:29 smithi189 ceph-mon[34851]: pgmap v389: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:33:30.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:29 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/274272358' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T07:33:30.577 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:31.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.212+0000 7f5675a00640 1 -- 172.21.15.159:0/1778426980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5670103e00 msgr2=0x7f5670104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:31.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.212+0000 7f5675a00640 1 --2- 172.21.15.159:0/1778426980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5670103e00 0x7f5670104200 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5660009900 tx=0x7f566002a410 comp rx=0 tx=0).stop 2024-09-19T07:33:31.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.212+0000 7f5675a00640 1 -- 172.21.15.159:0/1778426980 shutdown_connections 2024-09-19T07:33:31.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.212+0000 7f5675a00640 1 --2- 172.21.15.159:0/1778426980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5670105920 0x7f567010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:31.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.212+0000 7f5675a00640 1 --2- 172.21.15.159:0/1778426980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5670104f60 0x7f56701053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:31.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.212+0000 7f5675a00640 1 --2- 172.21.15.159:0/1778426980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5670103e00 0x7f5670104200 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:31.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.212+0000 7f5675a00640 1 -- 172.21.15.159:0/1778426980 >> 172.21.15.159:0/1778426980 conn(0x7f56700ff5b0 msgr2=0x7f56701019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:31.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.212+0000 7f5675a00640 1 -- 172.21.15.159:0/1778426980 shutdown_connections 2024-09-19T07:33:31.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.212+0000 7f5675a00640 1 -- 172.21.15.159:0/1778426980 wait complete. 2024-09-19T07:33:31.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.213+0000 7f5675a00640 1 Processor -- start 2024-09-19T07:33:31.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.213+0000 7f5675a00640 1 -- start start 2024-09-19T07:33:31.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.214+0000 7f5675a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5670103e00 0x7f567019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:31.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.214+0000 7f5675a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5670104f60 0x7f567019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:31.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.214+0000 7f5675a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5670105920 0x7f56701a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:31.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.214+0000 7f5675a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f567010fb00 con 0x7f5670104f60 2024-09-19T07:33:31.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.214+0000 7f5675a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f567010f980 con 0x7f5670103e00 2024-09-19T07:33:31.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.214+0000 7f5675a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f567010fc80 con 0x7f5670105920 2024-09-19T07:33:31.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.214+0000 7f566e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5670104f60 0x7f567019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:31.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.214+0000 7f566ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5670103e00 0x7f567019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:31.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.214+0000 7f566ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5670103e00 0x7f567019a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:43646/0 (socket says 172.21.15.159:43646) 2024-09-19T07:33:31.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.214+0000 7f566e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5670104f60 0x7f567019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47046/0 (socket says 172.21.15.159:47046) 2024-09-19T07:33:31.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.214+0000 7f566ea00640 1 -- 172.21.15.159:0/1909456747 learned_addr learned my addr 172.21.15.159:0/1909456747 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:31.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.214+0000 7f566f400640 1 --2- 172.21.15.159:0/1909456747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5670105920 0x7f56701a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:31.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.214+0000 7f566ea00640 1 -- 172.21.15.159:0/1909456747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5670105920 msgr2=0x7f56701a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:31.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.214+0000 7f566ea00640 1 --2- 172.21.15.159:0/1909456747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5670105920 0x7f56701a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:31.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.214+0000 7f566ea00640 1 -- 172.21.15.159:0/1909456747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5670104f60 msgr2=0x7f567019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:31.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.214+0000 7f566ea00640 1 --2- 172.21.15.159:0/1909456747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5670104f60 0x7f567019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:31.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.215+0000 7f566ea00640 1 -- 172.21.15.159:0/1909456747 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56701a1cf0 con 0x7f5670103e00 2024-09-19T07:33:31.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.215+0000 7f566f400640 1 --2- 172.21.15.159:0/1909456747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5670105920 0x7f56701a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:33:31.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.215+0000 7f566ea00640 1 --2- 172.21.15.159:0/1909456747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5670103e00 0x7f567019a0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5660031440 tx=0x7f5660032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:31.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.215+0000 7f5667400640 1 -- 172.21.15.159:0/1909456747 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566003f030 con 0x7f5670103e00 2024-09-19T07:33:31.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.215+0000 7f5675a00640 1 -- 172.21.15.159:0/1909456747 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56701a1f80 con 0x7f5670103e00 2024-09-19T07:33:31.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.215+0000 7f5667400640 1 -- 172.21.15.159:0/1909456747 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5660003ce0 con 0x7f5670103e00 2024-09-19T07:33:31.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.215+0000 7f5667400640 1 -- 172.21.15.159:0/1909456747 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660035070 con 0x7f5670103e00 2024-09-19T07:33:31.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.215+0000 7f5675a00640 1 -- 172.21.15.159:0/1909456747 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56701a23d0 con 0x7f5670103e00 2024-09-19T07:33:31.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.216+0000 7f5675a00640 1 -- 172.21.15.159:0/1909456747 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f567010b1a0 con 0x7f5670103e00 2024-09-19T07:33:31.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.218+0000 7f5667400640 1 -- 172.21.15.159:0/1909456747 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f5660042030 con 0x7f5670103e00 2024-09-19T07:33:31.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.220+0000 7f5667400640 1 --2- 172.21.15.159:0/1909456747 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f56540a3270 0x7f56540a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:31.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.220+0000 7f5667400640 1 -- 172.21.15.159:0/1909456747 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f56600f0660 con 0x7f5670103e00 2024-09-19T07:33:31.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.220+0000 7f566e000640 1 --2- 172.21.15.159:0/1909456747 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f56540a3270 0x7f56540a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:31.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.220+0000 7f566e000640 1 --2- 172.21.15.159:0/1909456747 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f56540a3270 0x7f56540a5730 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f567019b5e0 tx=0x7f565c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:31.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.222+0000 7f5667400640 1 -- 172.21.15.159:0/1909456747 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f56600bcf70 con 0x7f5670103e00 2024-09-19T07:33:31.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.401+0000 7f5675a00640 1 -- 172.21.15.159:0/1909456747 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f567002d9e0 con 0x7f5670103e00 2024-09-19T07:33:31.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.403+0000 7f5667400640 1 -- 172.21.15.159:0/1909456747 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v100) ==== 56+0+5711 (secure 0 0 0) 0x7f56600bc870 con 0x7f5670103e00 2024-09-19T07:33:31.403 INFO:teuthology.orchestra.run.smithi159.stdout:epoch 100 2024-09-19T07:33:31.403 INFO:teuthology.orchestra.run.smithi159.stdout:fsid c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:33:31.403 INFO:teuthology.orchestra.run.smithi159.stdout:created 2024-09-19T07:22:32.059836+0000 2024-09-19T07:33:31.403 INFO:teuthology.orchestra.run.smithi159.stdout:modified 2024-09-19T07:32:31.832496+0000 2024-09-19T07:33:31.403 INFO:teuthology.orchestra.run.smithi159.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T07:33:31.403 INFO:teuthology.orchestra.run.smithi159.stdout:crush_version 27 2024-09-19T07:33:31.403 INFO:teuthology.orchestra.run.smithi159.stdout:full_ratio 0.95 2024-09-19T07:33:31.403 INFO:teuthology.orchestra.run.smithi159.stdout:backfillfull_ratio 0.9 2024-09-19T07:33:31.403 INFO:teuthology.orchestra.run.smithi159.stdout:nearfull_ratio 0.85 2024-09-19T07:33:31.403 INFO:teuthology.orchestra.run.smithi159.stdout:require_min_compat_client luminous 2024-09-19T07:33:31.404 INFO:teuthology.orchestra.run.smithi159.stdout:min_compat_client luminous 2024-09-19T07:33:31.404 INFO:teuthology.orchestra.run.smithi159.stdout:require_osd_release squid 2024-09-19T07:33:31.404 INFO:teuthology.orchestra.run.smithi159.stdout:stretch_mode_enabled false 2024-09-19T07:33:31.404 INFO:teuthology.orchestra.run.smithi159.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-19T07:33:31.404 INFO:teuthology.orchestra.run.smithi159.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-19T07:33:31.404 INFO:teuthology.orchestra.run.smithi159.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-19T07:33:31.404 INFO:teuthology.orchestra.run.smithi159.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 96 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-19T07:33:31.404 INFO:teuthology.orchestra.run.smithi159.stdout:max_osd 12 2024-09-19T07:33:31.404 INFO:teuthology.orchestra.run.smithi159.stdout:osd.0 up in weight 1 up_from 8 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6802/2738900830,v1:172.21.15.159:6803/2738900830] [v2:172.21.15.159:6804/2738900830,v1:172.21.15.159:6805/2738900830] exists,up 752f5466-fe04-408a-8566-86625947fd3b 2024-09-19T07:33:31.404 INFO:teuthology.orchestra.run.smithi159.stdout:osd.1 up in weight 1 up_from 13 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6800/265696178,v1:172.21.15.189:6801/265696178] [v2:172.21.15.189:6802/265696178,v1:172.21.15.189:6803/265696178] exists,up 7513adda-ca82-4df4-aa99-3b050b410424 2024-09-19T07:33:31.404 INFO:teuthology.orchestra.run.smithi159.stdout:osd.2 up in weight 1 up_from 18 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6800/1826565288,v1:172.21.15.203:6801/1826565288] [v2:172.21.15.203:6802/1826565288,v1:172.21.15.203:6803/1826565288] exists,up aa03be79-58d8-4eb7-a051-0b46c502b12d 2024-09-19T07:33:31.404 INFO:teuthology.orchestra.run.smithi159.stdout:osd.3 up in weight 1 up_from 26 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6810/109457380,v1:172.21.15.159:6811/109457380] [v2:172.21.15.159:6812/109457380,v1:172.21.15.159:6813/109457380] exists,up fcd734f1-f40f-4566-82ab-c4607039f23f 2024-09-19T07:33:31.404 INFO:teuthology.orchestra.run.smithi159.stdout:osd.4 up in weight 1 up_from 34 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6808/1036888180,v1:172.21.15.189:6809/1036888180] [v2:172.21.15.189:6810/1036888180,v1:172.21.15.189:6811/1036888180] exists,up bc344c9e-2267-4058-bbe1-9c270327db8a 2024-09-19T07:33:31.404 INFO:teuthology.orchestra.run.smithi159.stdout:osd.5 up in weight 1 up_from 39 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6808/4212601512,v1:172.21.15.203:6809/4212601512] [v2:172.21.15.203:6810/4212601512,v1:172.21.15.203:6811/4212601512] exists,up 86c48303-929d-42f4-bf48-cbb32649f95f 2024-09-19T07:33:31.405 INFO:teuthology.orchestra.run.smithi159.stdout:osd.6 up in weight 1 up_from 47 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6818/3817684578,v1:172.21.15.159:6819/3817684578] [v2:172.21.15.159:6820/3817684578,v1:172.21.15.159:6821/3817684578] exists,up f0176ed1-c3a7-4d75-b213-fe88fab4441a 2024-09-19T07:33:31.405 INFO:teuthology.orchestra.run.smithi159.stdout:osd.7 up in weight 1 up_from 52 up_thru 99 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6816/3936826456,v1:172.21.15.189:6817/3936826456] [v2:172.21.15.189:6818/3936826456,v1:172.21.15.189:6819/3936826456] exists,up 921bd59d-9415-4559-8e9f-035d336afd65 2024-09-19T07:33:31.405 INFO:teuthology.orchestra.run.smithi159.stdout:osd.8 up in weight 1 up_from 61 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6816/4062926645,v1:172.21.15.203:6817/4062926645] [v2:172.21.15.203:6818/4062926645,v1:172.21.15.203:6819/4062926645] exists,up ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3 2024-09-19T07:33:31.405 INFO:teuthology.orchestra.run.smithi159.stdout:osd.9 up in weight 1 up_from 69 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6826/4185848747,v1:172.21.15.159:6827/4185848747] [v2:172.21.15.159:6828/4185848747,v1:172.21.15.159:6829/4185848747] exists,up df29a10f-1c60-41cb-9c2d-bc9f1cb174bc 2024-09-19T07:33:31.405 INFO:teuthology.orchestra.run.smithi159.stdout:osd.10 up in weight 1 up_from 74 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6824/3018377605,v1:172.21.15.189:6825/3018377605] [v2:172.21.15.189:6826/3018377605,v1:172.21.15.189:6827/3018377605] exists,up ccc440b4-4682-4998-bb53-95e24423cd1b 2024-09-19T07:33:31.405 INFO:teuthology.orchestra.run.smithi159.stdout:osd.11 up in weight 1 up_from 81 up_thru 99 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6824/2807503891,v1:172.21.15.203:6825/2807503891] [v2:172.21.15.203:6826/2807503891,v1:172.21.15.203:6827/2807503891] exists,up e84afc5f-8051-4341-8fd0-3afb482ab4df 2024-09-19T07:33:31.405 INFO:teuthology.orchestra.run.smithi159.stdout:pg_upmap_items 2.0 [8,4] 2024-09-19T07:33:31.405 INFO:teuthology.orchestra.run.smithi159.stdout:pg_upmap_items 2.e [2,4] 2024-09-19T07:33:31.405 INFO:teuthology.orchestra.run.smithi159.stdout:pg_upmap_items 2.f [8,11] 2024-09-19T07:33:31.405 INFO:teuthology.orchestra.run.smithi159.stdout:pg_upmap_items 2.13 [2,11] 2024-09-19T07:33:31.405 INFO:teuthology.orchestra.run.smithi159.stdout:pg_upmap_items 3.0 [4,6] 2024-09-19T07:33:31.405 INFO:teuthology.orchestra.run.smithi159.stdout:pg_upmap_items 3.1b [5,9] 2024-09-19T07:33:31.405 INFO:teuthology.orchestra.run.smithi159.stdout:pg_upmap_items 3.1d [4,8] 2024-09-19T07:33:31.405 INFO:teuthology.orchestra.run.smithi159.stdout:pg_upmap_items 3.24 [4,9] 2024-09-19T07:33:31.406 INFO:teuthology.orchestra.run.smithi159.stdout:pg_upmap_items 3.27 [3,9] 2024-09-19T07:33:31.406 INFO:teuthology.orchestra.run.smithi159.stdout:pg_upmap_items 4.f [9,10] 2024-09-19T07:33:31.406 INFO:teuthology.orchestra.run.smithi159.stdout:pg_upmap_items 4.13 [9,8] 2024-09-19T07:33:31.406 INFO:teuthology.orchestra.run.smithi159.stdout:pg_upmap_items 4.14 [9,8] 2024-09-19T07:33:31.406 INFO:teuthology.orchestra.run.smithi159.stdout:pg_upmap_items 4.15 [5,10] 2024-09-19T07:33:31.406 INFO:teuthology.orchestra.run.smithi159.stdout:blocklist 172.21.15.159:0/2383222735 expires 2024-09-20T07:23:30.596931+0000 2024-09-19T07:33:31.406 INFO:teuthology.orchestra.run.smithi159.stdout:blocklist 172.21.15.159:0/1561371050 expires 2024-09-20T07:23:30.596931+0000 2024-09-19T07:33:31.406 INFO:teuthology.orchestra.run.smithi159.stdout:blocklist 172.21.15.159:6801/2966947684 expires 2024-09-20T07:23:30.596931+0000 2024-09-19T07:33:31.406 INFO:teuthology.orchestra.run.smithi159.stdout:blocklist 172.21.15.159:6800/2966947684 expires 2024-09-20T07:23:30.596931+0000 2024-09-19T07:33:31.406 INFO:teuthology.orchestra.run.smithi159.stdout:blocklist 172.21.15.159:6801/432893324 expires 2024-09-20T07:23:03.089190+0000 2024-09-19T07:33:31.406 INFO:teuthology.orchestra.run.smithi159.stdout:blocklist 172.21.15.159:6800/432893324 expires 2024-09-20T07:23:03.089190+0000 2024-09-19T07:33:31.406 INFO:teuthology.orchestra.run.smithi159.stdout:blocklist 172.21.15.159:0/2108730247 expires 2024-09-20T07:23:03.089190+0000 2024-09-19T07:33:31.406 INFO:teuthology.orchestra.run.smithi159.stdout:blocklist 172.21.15.159:0/3393992615 expires 2024-09-20T07:23:30.596931+0000 2024-09-19T07:33:31.406 INFO:teuthology.orchestra.run.smithi159.stdout:blocklist 172.21.15.159:0/1366863724 expires 2024-09-20T07:23:03.089190+0000 2024-09-19T07:33:31.406 INFO:teuthology.orchestra.run.smithi159.stdout:blocklist 172.21.15.159:0/1628344860 expires 2024-09-20T07:23:03.089190+0000 2024-09-19T07:33:31.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.405+0000 7f5675a00640 1 -- 172.21.15.159:0/1909456747 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f56540a3270 msgr2=0x7f56540a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:31.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.405+0000 7f5675a00640 1 --2- 172.21.15.159:0/1909456747 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f56540a3270 0x7f56540a5730 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f567019b5e0 tx=0x7f565c005f90 comp rx=0 tx=0).stop 2024-09-19T07:33:31.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.405+0000 7f5675a00640 1 -- 172.21.15.159:0/1909456747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5670103e00 msgr2=0x7f567019a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:31.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.405+0000 7f5675a00640 1 --2- 172.21.15.159:0/1909456747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5670103e00 0x7f567019a0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5660031440 tx=0x7f5660032ce0 comp rx=0 tx=0).stop 2024-09-19T07:33:31.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.405+0000 7f5675a00640 1 -- 172.21.15.159:0/1909456747 shutdown_connections 2024-09-19T07:33:31.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.405+0000 7f5675a00640 1 --2- 172.21.15.159:0/1909456747 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f56540a3270 0x7f56540a5730 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:31.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.405+0000 7f5675a00640 1 --2- 172.21.15.159:0/1909456747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5670105920 0x7f56701a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:31.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.405+0000 7f5675a00640 1 --2- 172.21.15.159:0/1909456747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5670104f60 0x7f567019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:31.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.405+0000 7f5675a00640 1 --2- 172.21.15.159:0/1909456747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5670103e00 0x7f567019a0c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:31.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.405+0000 7f5675a00640 1 -- 172.21.15.159:0/1909456747 >> 172.21.15.159:0/1909456747 conn(0x7f56700ff5b0 msgr2=0x7f5670108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:31.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.405+0000 7f5675a00640 1 -- 172.21.15.159:0/1909456747 shutdown_connections 2024-09-19T07:33:31.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:31.405+0000 7f5675a00640 1 -- 172.21.15.159:0/1909456747 wait complete. 2024-09-19T07:33:32.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:31 smithi159 ceph-mon[30472]: pgmap v390: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:32.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:31 smithi159 ceph-mon[30472]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-19T07:33:32.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:31 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1909456747' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T07:33:32.234 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T07:33:32.245 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:33:32.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:31 smithi203 ceph-mon[35335]: pgmap v390: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:32.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:31 smithi203 ceph-mon[35335]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-19T07:33:32.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:31 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1909456747' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T07:33:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:31 smithi189 ceph-mon[34851]: pgmap v390: 129 pgs: 129 active+clean; 641 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:33:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:31 smithi189 ceph-mon[34851]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-19T07:33:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:31 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1909456747' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T07:33:32.620 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:33.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.328+0000 7f7648c00640 1 -- 172.21.15.159:0/1729555241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7644102790 msgr2=0x7f76441067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:33.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.328+0000 7f7648c00640 1 --2- 172.21.15.159:0/1729555241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7644102790 0x7f76441067e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f762c009900 tx=0x7f762c02a3f0 comp rx=0 tx=0).stop 2024-09-19T07:33:33.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.328+0000 7f7648c00640 1 -- 172.21.15.159:0/1729555241 shutdown_connections 2024-09-19T07:33:33.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.328+0000 7f7648c00640 1 --2- 172.21.15.159:0/1729555241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7644106f10 0x7f764410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:33.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.328+0000 7f7648c00640 1 --2- 172.21.15.159:0/1729555241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7644102790 0x7f76441067e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:33.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.328+0000 7f7648c00640 1 --2- 172.21.15.159:0/1729555241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7644101de0 0x7f76441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:33.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.328+0000 7f7648c00640 1 -- 172.21.15.159:0/1729555241 >> 172.21.15.159:0/1729555241 conn(0x7f76440fd8c0 msgr2=0x7f76440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:33.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.328+0000 7f7648c00640 1 -- 172.21.15.159:0/1729555241 shutdown_connections 2024-09-19T07:33:33.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.328+0000 7f7648c00640 1 -- 172.21.15.159:0/1729555241 wait complete. 2024-09-19T07:33:33.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.329+0000 7f7648c00640 1 Processor -- start 2024-09-19T07:33:33.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.329+0000 7f7648c00640 1 -- start start 2024-09-19T07:33:33.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.329+0000 7f7648c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7644101de0 0x7f764419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:33.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.329+0000 7f7648c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7644102790 0x7f764419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:33.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.329+0000 7f7648c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7644106f10 0x7f764419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:33.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.329+0000 7f7648c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7644111f70 con 0x7f7644106f10 2024-09-19T07:33:33.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.329+0000 7f7648c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7644111df0 con 0x7f7644102790 2024-09-19T07:33:33.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.329+0000 7f7648c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f76441120f0 con 0x7f7644101de0 2024-09-19T07:33:33.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.330+0000 7f7641e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7644101de0 0x7f764419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:33.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.330+0000 7f7642800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7644106f10 0x7f764419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:33.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.330+0000 7f7641e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7644101de0 0x7f764419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:34548/0 (socket says 172.21.15.159:34548) 2024-09-19T07:33:33.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.330+0000 7f7641e00640 1 -- 172.21.15.159:0/2945566270 learned_addr learned my addr 172.21.15.159:0/2945566270 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:33.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.330+0000 7f7639400640 1 --2- 172.21.15.159:0/2945566270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7644102790 0x7f764419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:33.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.330+0000 7f7641e00640 1 -- 172.21.15.159:0/2945566270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7644102790 msgr2=0x7f764419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:33.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.330+0000 7f7641e00640 1 --2- 172.21.15.159:0/2945566270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7644102790 0x7f764419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:33.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.330+0000 7f7641e00640 1 -- 172.21.15.159:0/2945566270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7644106f10 msgr2=0x7f764419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:33.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.330+0000 7f7641e00640 1 --2- 172.21.15.159:0/2945566270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7644106f10 0x7f764419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:33.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.330+0000 7f7641e00640 1 -- 172.21.15.159:0/2945566270 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764419f600 con 0x7f7644101de0 2024-09-19T07:33:33.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.330+0000 7f7642800640 1 --2- 172.21.15.159:0/2945566270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7644106f10 0x7f764419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:33.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.330+0000 7f7641e00640 1 --2- 172.21.15.159:0/2945566270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7644101de0 0x7f764419a650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f763400eb90 tx=0x7f763400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:33.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.331+0000 7f763b400640 1 -- 172.21.15.159:0/2945566270 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634004d20 con 0x7f7644101de0 2024-09-19T07:33:33.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.331+0000 7f763b400640 1 -- 172.21.15.159:0/2945566270 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763400ee90 con 0x7f7644101de0 2024-09-19T07:33:33.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.331+0000 7f763b400640 1 -- 172.21.15.159:0/2945566270 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634010830 con 0x7f7644101de0 2024-09-19T07:33:33.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.331+0000 7f7648c00640 1 -- 172.21.15.159:0/2945566270 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764419f8f0 con 0x7f7644101de0 2024-09-19T07:33:33.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.331+0000 7f7648c00640 1 -- 172.21.15.159:0/2945566270 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76441a71d0 con 0x7f7644101de0 2024-09-19T07:33:33.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.332+0000 7f7648c00640 1 -- 172.21.15.159:0/2945566270 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76441021c0 con 0x7f7644101de0 2024-09-19T07:33:33.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.334+0000 7f763b400640 1 -- 172.21.15.159:0/2945566270 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f7634005020 con 0x7f7644101de0 2024-09-19T07:33:33.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.335+0000 7f763b400640 1 --2- 172.21.15.159:0/2945566270 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f76280a3240 0x7f76280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:33.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.335+0000 7f763b400640 1 -- 172.21.15.159:0/2945566270 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f76340d0600 con 0x7f7644101de0 2024-09-19T07:33:33.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.336+0000 7f7639400640 1 --2- 172.21.15.159:0/2945566270 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f76280a3240 0x7f76280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:33.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.336+0000 7f7639400640 1 --2- 172.21.15.159:0/2945566270 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f76280a3240 0x7f76280a5700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f762c009230 tx=0x7f762c0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:33.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.337+0000 7f763b400640 1 -- 172.21.15.159:0/2945566270 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f763409cf10 con 0x7f7644101de0 2024-09-19T07:33:33.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.550+0000 7f7648c00640 1 -- 172.21.15.159:0/2945566270 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f764419b9b0 con 0x7f7644101de0 2024-09-19T07:33:33.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.551+0000 7f763b400640 1 -- 172.21.15.159:0/2945566270 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11936 (secure 0 0 0) 0x7f763409c810 con 0x7f7644101de0 2024-09-19T07:33:33.551 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:33:33.552 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":25,"btime":"2024-09-19T07:31:54:895489+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3464449771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3464449771},{"type":"v1","addr":"172.21.15.159:6837","nonce":3464449771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2036264856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2036264856},{"type":"v1","addr":"172.21.15.159:6839","nonce":2036264856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6841/2874566405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6840","nonce":2874566405},{"type":"v1","addr":"172.21.15.159:6841","nonce":2874566405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:31:03.518840+0000","modified":"2024-09-19T07:31:53.954806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24542,"mds_1":24514,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/1340279549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1340279549},{"type":"v1","addr":"172.21.15.189:6835","nonce":1340279549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/937970878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":937970878},{"type":"v1","addr":"172.21.15.203:6835","nonce":937970878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1053443810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1053443810},{"type":"v1","addr":"172.21.15.203:6837","nonce":1053443810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.189:6837/1030825585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1030825585},{"type":"v1","addr":"172.21.15.189:6837","nonce":1030825585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.203:6839/3862781462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":3862781462},{"type":"v1","addr":"172.21.15.203:6839","nonce":3862781462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.189:6839/2361818663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2361818663},{"type":"v1","addr":"172.21.15.189:6839","nonce":2361818663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24506,24514,24542]},"id":1}]} 2024-09-19T07:33:33.552 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 25 2024-09-19T07:33:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.553+0000 7f7648c00640 1 -- 172.21.15.159:0/2945566270 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f76280a3240 msgr2=0x7f76280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.554+0000 7f7648c00640 1 --2- 172.21.15.159:0/2945566270 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f76280a3240 0x7f76280a5700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f762c009230 tx=0x7f762c0058d0 comp rx=0 tx=0).stop 2024-09-19T07:33:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.554+0000 7f7648c00640 1 -- 172.21.15.159:0/2945566270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7644101de0 msgr2=0x7f764419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.554+0000 7f7648c00640 1 --2- 172.21.15.159:0/2945566270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7644101de0 0x7f764419a650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f763400eb90 tx=0x7f763400c6a0 comp rx=0 tx=0).stop 2024-09-19T07:33:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.554+0000 7f7648c00640 1 -- 172.21.15.159:0/2945566270 shutdown_connections 2024-09-19T07:33:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.554+0000 7f7648c00640 1 --2- 172.21.15.159:0/2945566270 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f76280a3240 0x7f76280a5700 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.554+0000 7f7648c00640 1 --2- 172.21.15.159:0/2945566270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7644106f10 0x7f764419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.554+0000 7f7648c00640 1 --2- 172.21.15.159:0/2945566270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7644102790 0x7f764419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.554+0000 7f7648c00640 1 --2- 172.21.15.159:0/2945566270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7644101de0 0x7f764419a650 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:33.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.554+0000 7f7648c00640 1 -- 172.21.15.159:0/2945566270 >> 172.21.15.159:0/2945566270 conn(0x7f76440fd8c0 msgr2=0x7f76440feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:33.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.554+0000 7f7648c00640 1 -- 172.21.15.159:0/2945566270 shutdown_connections 2024-09-19T07:33:33.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:33.554+0000 7f7648c00640 1 -- 172.21.15.159:0/2945566270 wait complete. 2024-09-19T07:33:34.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:33 smithi159 ceph-mon[30472]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T07:33:34.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:33 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2945566270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:33:34.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:33 smithi203 ceph-mon[35335]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T07:33:34.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:33 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/2945566270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:33:34.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:33 smithi189 ceph-mon[34851]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T07:33:34.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:33 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/2945566270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:33:34.403 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:33:34.776 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:35.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.353+0000 7fbff5600640 1 -- 172.21.15.159:0/452219196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff010a3c0 msgr2=0x7fbff0110d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:35.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.353+0000 7fbff5600640 1 --2- 172.21.15.159:0/452219196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff010a3c0 0x7fbff0110d40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009ca0 tx=0x7fbfe802a8b0 comp rx=0 tx=0).stop 2024-09-19T07:33:35.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.353+0000 7fbff5600640 1 -- 172.21.15.159:0/452219196 shutdown_connections 2024-09-19T07:33:35.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.353+0000 7fbff5600640 1 --2- 172.21.15.159:0/452219196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff010a3c0 0x7fbff0110d40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:35.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.353+0000 7fbff5600640 1 --2- 172.21.15.159:0/452219196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbff0100850 0x7fbff0109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:35.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.353+0000 7fbff5600640 1 --2- 172.21.15.159:0/452219196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbff00ffea0 0x7fbff0100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:35.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.353+0000 7fbff5600640 1 -- 172.21.15.159:0/452219196 >> 172.21.15.159:0/452219196 conn(0x7fbff00fba90 msgr2=0x7fbff00fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:35.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.353+0000 7fbff5600640 1 -- 172.21.15.159:0/452219196 shutdown_connections 2024-09-19T07:33:35.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.353+0000 7fbff5600640 1 -- 172.21.15.159:0/452219196 wait complete. 2024-09-19T07:33:35.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.354+0000 7fbff5600640 1 Processor -- start 2024-09-19T07:33:35.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.354+0000 7fbff5600640 1 -- start start 2024-09-19T07:33:35.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.355+0000 7fbff5600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbff00ffea0 0x7fbff019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:35.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.355+0000 7fbff5600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff0100850 0x7fbff019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:35.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.355+0000 7fbff5600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbff010a3c0 0x7fbff019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:35.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.355+0000 7fbff5600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0114180 con 0x7fbff0100850 2024-09-19T07:33:35.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.355+0000 7fbff5600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0114000 con 0x7fbff010a3c0 2024-09-19T07:33:35.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.355+0000 7fbff5600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0114300 con 0x7fbff00ffea0 2024-09-19T07:33:35.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.355+0000 7fbfee000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff0100850 0x7fbff019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:35.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.355+0000 7fbfeea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbff00ffea0 0x7fbff019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:35.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.355+0000 7fbfef400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbff010a3c0 0x7fbff019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:35.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.355+0000 7fbfeea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbff00ffea0 0x7fbff019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:34580/0 (socket says 172.21.15.159:34580) 2024-09-19T07:33:35.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.355+0000 7fbfee000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff0100850 0x7fbff019aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37358/0 (socket says 172.21.15.159:37358) 2024-09-19T07:33:35.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.355+0000 7fbfeea00640 1 -- 172.21.15.159:0/1541644953 learned_addr learned my addr 172.21.15.159:0/1541644953 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:35.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.355+0000 7fbfef400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbff010a3c0 0x7fbff019ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:58188/0 (socket says 172.21.15.159:58188) 2024-09-19T07:33:35.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.356+0000 7fbfef400640 1 -- 172.21.15.159:0/1541644953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbff00ffea0 msgr2=0x7fbff019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:35.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.356+0000 7fbfef400640 1 --2- 172.21.15.159:0/1541644953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbff00ffea0 0x7fbff019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:35.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.356+0000 7fbfef400640 1 -- 172.21.15.159:0/1541644953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff0100850 msgr2=0x7fbff019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:35.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.356+0000 7fbfef400640 1 --2- 172.21.15.159:0/1541644953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff0100850 0x7fbff019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:35.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.356+0000 7fbfef400640 1 -- 172.21.15.159:0/1541644953 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff019f550 con 0x7fbff010a3c0 2024-09-19T07:33:35.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.356+0000 7fbfef400640 1 --2- 172.21.15.159:0/1541644953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbff010a3c0 0x7fbff019ee70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8005e30 tx=0x7fbfe8003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:35.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.356+0000 7fbfe7400640 1 -- 172.21.15.159:0/1541644953 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8030dc0 con 0x7fbff010a3c0 2024-09-19T07:33:35.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.356+0000 7fbff5600640 1 -- 172.21.15.159:0/1541644953 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff019f7e0 con 0x7fbff010a3c0 2024-09-19T07:33:35.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.356+0000 7fbfe7400640 1 -- 172.21.15.159:0/1541644953 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe8040070 con 0x7fbff010a3c0 2024-09-19T07:33:35.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.356+0000 7fbfe7400640 1 -- 172.21.15.159:0/1541644953 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8045a70 con 0x7fbff010a3c0 2024-09-19T07:33:35.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.356+0000 7fbff5600640 1 -- 172.21.15.159:0/1541644953 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff01a7120 con 0x7fbff010a3c0 2024-09-19T07:33:35.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.358+0000 7fbff5600640 1 -- 172.21.15.159:0/1541644953 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff0100280 con 0x7fbff010a3c0 2024-09-19T07:33:35.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.359+0000 7fbfe7400640 1 -- 172.21.15.159:0/1541644953 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fbfe8004700 con 0x7fbff010a3c0 2024-09-19T07:33:35.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.361+0000 7fbfe7400640 1 --2- 172.21.15.159:0/1541644953 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fbfd40a3240 0x7fbfd40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:35.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.361+0000 7fbfe7400640 1 -- 172.21.15.159:0/1541644953 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fbfe80f0c00 con 0x7fbff010a3c0 2024-09-19T07:33:35.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.361+0000 7fbfeea00640 1 --2- 172.21.15.159:0/1541644953 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fbfd40a3240 0x7fbfd40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:35.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.362+0000 7fbfeea00640 1 --2- 172.21.15.159:0/1541644953 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fbfd40a3240 0x7fbfd40a5700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009c90 tx=0x7fbfe0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:35.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.365+0000 7fbfe7400640 1 -- 172.21.15.159:0/1541644953 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbfe80bd5a0 con 0x7fbff010a3c0 2024-09-19T07:33:35.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.580+0000 7fbff5600640 1 -- 172.21.15.159:0/1541644953 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbff01a72f0 con 0x7fbff010a3c0 2024-09-19T07:33:35.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.581+0000 7fbfe7400640 1 -- 172.21.15.159:0/1541644953 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11936 (secure 0 0 0) 0x7fbfe80bcea0 con 0x7fbff010a3c0 2024-09-19T07:33:35.581 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:33:35.582 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":25,"btime":"2024-09-19T07:31:54:895489+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3464449771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3464449771},{"type":"v1","addr":"172.21.15.159:6837","nonce":3464449771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2036264856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2036264856},{"type":"v1","addr":"172.21.15.159:6839","nonce":2036264856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6841/2874566405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6840","nonce":2874566405},{"type":"v1","addr":"172.21.15.159:6841","nonce":2874566405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:31:03.518840+0000","modified":"2024-09-19T07:31:53.954806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24542,"mds_1":24514,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/1340279549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1340279549},{"type":"v1","addr":"172.21.15.189:6835","nonce":1340279549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/937970878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":937970878},{"type":"v1","addr":"172.21.15.203:6835","nonce":937970878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1053443810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1053443810},{"type":"v1","addr":"172.21.15.203:6837","nonce":1053443810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.189:6837/1030825585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1030825585},{"type":"v1","addr":"172.21.15.189:6837","nonce":1030825585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.203:6839/3862781462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":3862781462},{"type":"v1","addr":"172.21.15.203:6839","nonce":3862781462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.189:6839/2361818663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2361818663},{"type":"v1","addr":"172.21.15.189:6839","nonce":2361818663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24506,24514,24542]},"id":1}]} 2024-09-19T07:33:35.582 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 25 2024-09-19T07:33:35.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.584+0000 7fbff5600640 1 -- 172.21.15.159:0/1541644953 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fbfd40a3240 msgr2=0x7fbfd40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:35.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.584+0000 7fbff5600640 1 --2- 172.21.15.159:0/1541644953 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fbfd40a3240 0x7fbfd40a5700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009c90 tx=0x7fbfe0005f90 comp rx=0 tx=0).stop 2024-09-19T07:33:35.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.584+0000 7fbff5600640 1 -- 172.21.15.159:0/1541644953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbff010a3c0 msgr2=0x7fbff019ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:35.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.584+0000 7fbff5600640 1 --2- 172.21.15.159:0/1541644953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbff010a3c0 0x7fbff019ee70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8005e30 tx=0x7fbfe8003f10 comp rx=0 tx=0).stop 2024-09-19T07:33:35.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.585+0000 7fbff5600640 1 -- 172.21.15.159:0/1541644953 shutdown_connections 2024-09-19T07:33:35.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.585+0000 7fbff5600640 1 --2- 172.21.15.159:0/1541644953 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fbfd40a3240 0x7fbfd40a5700 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:35.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.585+0000 7fbff5600640 1 --2- 172.21.15.159:0/1541644953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbff010a3c0 0x7fbff019ee70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:35.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.585+0000 7fbff5600640 1 --2- 172.21.15.159:0/1541644953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff0100850 0x7fbff019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:35.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.585+0000 7fbff5600640 1 --2- 172.21.15.159:0/1541644953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbff00ffea0 0x7fbff019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:35.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.585+0000 7fbff5600640 1 -- 172.21.15.159:0/1541644953 >> 172.21.15.159:0/1541644953 conn(0x7fbff00fba90 msgr2=0x7fbff0108dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:35.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.585+0000 7fbff5600640 1 -- 172.21.15.159:0/1541644953 shutdown_connections 2024-09-19T07:33:35.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:35.585+0000 7fbff5600640 1 -- 172.21.15.159:0/1541644953 wait complete. 2024-09-19T07:33:36.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:35 smithi159 ceph-mon[30472]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T07:33:36.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:35 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1541644953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:33:36.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:35 smithi203 ceph-mon[35335]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T07:33:36.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:35 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1541644953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:33:36.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:35 smithi189 ceph-mon[34851]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T07:33:36.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:35 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1541644953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:33:36.548 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:33:36.928 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:37.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.571+0000 7fcff1400640 1 -- 172.21.15.159:0/4117623613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfec074f40 msgr2=0x7fcfec10dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:37.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.571+0000 7fcff1400640 1 --2- 172.21.15.159:0/4117623613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfec074f40 0x7fcfec10dcc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009900 tx=0x7fcfd802a480 comp rx=0 tx=0).stop 2024-09-19T07:33:37.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.571+0000 7fcff1400640 1 -- 172.21.15.159:0/4117623613 shutdown_connections 2024-09-19T07:33:37.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.571+0000 7fcff1400640 1 --2- 172.21.15.159:0/4117623613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfec10e200 0x7fcfec110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:37.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.571+0000 7fcff1400640 1 --2- 172.21.15.159:0/4117623613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfec074f40 0x7fcfec10dcc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:37.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.571+0000 7fcff1400640 1 --2- 172.21.15.159:0/4117623613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfec074620 0x7fcfec074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:37.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.571+0000 7fcff1400640 1 -- 172.21.15.159:0/4117623613 >> 172.21.15.159:0/4117623613 conn(0x7fcfec0fd6e0 msgr2=0x7fcfec0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:37.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.571+0000 7fcff1400640 1 -- 172.21.15.159:0/4117623613 shutdown_connections 2024-09-19T07:33:37.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.571+0000 7fcff1400640 1 -- 172.21.15.159:0/4117623613 wait complete. 2024-09-19T07:33:37.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.572+0000 7fcff1400640 1 Processor -- start 2024-09-19T07:33:37.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.572+0000 7fcff1400640 1 -- start start 2024-09-19T07:33:37.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.573+0000 7fcff1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfec074620 0x7fcfec19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:37.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.573+0000 7fcff1400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfec074f40 0x7fcfec19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:37.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.573+0000 7fcff1400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfec10e200 0x7fcfec19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:37.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.573+0000 7fcff1400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec113fb0 con 0x7fcfec074620 2024-09-19T07:33:37.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.573+0000 7fcff1400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec113e30 con 0x7fcfec074f40 2024-09-19T07:33:37.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.573+0000 7fcff1400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec114130 con 0x7fcfec10e200 2024-09-19T07:33:37.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.573+0000 7fcfebe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfec074620 0x7fcfec19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:37.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.573+0000 7fcfebe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfec074620 0x7fcfec19a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37378/0 (socket says 172.21.15.159:37378) 2024-09-19T07:33:37.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.573+0000 7fcfebe00640 1 -- 172.21.15.159:0/3150537324 learned_addr learned my addr 172.21.15.159:0/3150537324 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:37.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.573+0000 7fcfeb400640 1 --2- 172.21.15.159:0/3150537324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfec074f40 0x7fcfec19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:37.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.573+0000 7fcff0a00640 1 --2- 172.21.15.159:0/3150537324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfec10e200 0x7fcfec19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:37.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.573+0000 7fcfebe00640 1 -- 172.21.15.159:0/3150537324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfec10e200 msgr2=0x7fcfec19ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:37.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.573+0000 7fcfebe00640 1 --2- 172.21.15.159:0/3150537324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfec10e200 0x7fcfec19ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:37.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.573+0000 7fcfebe00640 1 -- 172.21.15.159:0/3150537324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfec074f40 msgr2=0x7fcfec19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:37.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.573+0000 7fcfebe00640 1 --2- 172.21.15.159:0/3150537324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfec074f40 0x7fcfec19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:37.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.573+0000 7fcfebe00640 1 -- 172.21.15.159:0/3150537324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfec19f4c0 con 0x7fcfec074620 2024-09-19T07:33:37.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.574+0000 7fcfebe00640 1 --2- 172.21.15.159:0/3150537324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfec074620 0x7fcfec19a510 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc00e9f0 tx=0x7fcfdc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:37.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.574+0000 7fcfe8c00640 1 -- 172.21.15.159:0/3150537324 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc00cd80 con 0x7fcfec074620 2024-09-19T07:33:37.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.574+0000 7fcff1400640 1 -- 172.21.15.159:0/3150537324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfec19f7b0 con 0x7fcfec074620 2024-09-19T07:33:37.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.574+0000 7fcfe8c00640 1 -- 172.21.15.159:0/3150537324 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfdc005500 con 0x7fcfec074620 2024-09-19T07:33:37.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.574+0000 7fcfe8c00640 1 -- 172.21.15.159:0/3150537324 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc010690 con 0x7fcfec074620 2024-09-19T07:33:37.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.574+0000 7fcff1400640 1 -- 172.21.15.159:0/3150537324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfec1a7090 con 0x7fcfec074620 2024-09-19T07:33:37.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.575+0000 7fcff1400640 1 -- 172.21.15.159:0/3150537324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfec074a00 con 0x7fcfec074620 2024-09-19T07:33:37.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.577+0000 7fcfe8c00640 1 -- 172.21.15.159:0/3150537324 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fcfdc0046c0 con 0x7fcfec074620 2024-09-19T07:33:37.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.578+0000 7fcfe8c00640 1 --2- 172.21.15.159:0/3150537324 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcfd00a32c0 0x7fcfd00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:37.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.579+0000 7fcfe8c00640 1 -- 172.21.15.159:0/3150537324 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fcfdc0d00a0 con 0x7fcfec074620 2024-09-19T07:33:37.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.579+0000 7fcfeb400640 1 --2- 172.21.15.159:0/3150537324 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcfd00a32c0 0x7fcfd00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:37.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.580+0000 7fcfeb400640 1 --2- 172.21.15.159:0/3150537324 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcfd00a32c0 0x7fcfd00a5780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009230 tx=0x7fcfd80058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:37.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.583+0000 7fcfe8c00640 1 -- 172.21.15.159:0/3150537324 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcfdc09c9b0 con 0x7fcfec074620 2024-09-19T07:33:37.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.758+0000 7fcff1400640 1 -- 172.21.15.159:0/3150537324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcfec1093e0 con 0x7fcfec074620 2024-09-19T07:33:37.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.759+0000 7fcfe8c00640 1 -- 172.21.15.159:0/3150537324 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24963 (secure 0 0 0) 0x7fcfdc096e20 con 0x7fcfec074620 2024-09-19T07:33:37.760 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:33:37.760 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":100,"fsid":"c9d18048-7657-11ef-bceb-c7b262605968","created":"2024-09-19T07:22:32.059836+0000","modified":"2024-09-19T07:32:31.832496+0000","last_up_change":"2024-09-19T07:30:03.328697+0000","last_in_change":"2024-09-19T07:29:38.987579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:25:55.202457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:30:58.022056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:31:01.042290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:31:10.977289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"752f5466-fe04-408a-8566-86625947fd3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6803","nonce":2738900830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6805","nonce":2738900830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6809","nonce":2738900830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6807","nonce":2738900830}]},"public_addr":"172.21.15.159:6803/2738900830","cluster_addr":"172.21.15.159:6805/2738900830","heartbeat_back_addr":"172.21.15.159:6809/2738900830","heartbeat_front_addr":"172.21.15.159:6807/2738900830","state":["exists","up"]},{"osd":1,"uuid":"7513adda-ca82-4df4-aa99-3b050b410424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6801","nonce":265696178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6803","nonce":265696178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6807","nonce":265696178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6805","nonce":265696178}]},"public_addr":"172.21.15.189:6801/265696178","cluster_addr":"172.21.15.189:6803/265696178","heartbeat_back_addr":"172.21.15.189:6807/265696178","heartbeat_front_addr":"172.21.15.189:6805/265696178","state":["exists","up"]},{"osd":2,"uuid":"aa03be79-58d8-4eb7-a051-0b46c502b12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6801","nonce":1826565288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6803","nonce":1826565288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6807","nonce":1826565288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6805","nonce":1826565288}]},"public_addr":"172.21.15.203:6801/1826565288","cluster_addr":"172.21.15.203:6803/1826565288","heartbeat_back_addr":"172.21.15.203:6807/1826565288","heartbeat_front_addr":"172.21.15.203:6805/1826565288","state":["exists","up"]},{"osd":3,"uuid":"fcd734f1-f40f-4566-82ab-c4607039f23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6811","nonce":109457380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6813","nonce":109457380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6817","nonce":109457380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6815","nonce":109457380}]},"public_addr":"172.21.15.159:6811/109457380","cluster_addr":"172.21.15.159:6813/109457380","heartbeat_back_addr":"172.21.15.159:6817/109457380","heartbeat_front_addr":"172.21.15.159:6815/109457380","state":["exists","up"]},{"osd":4,"uuid":"bc344c9e-2267-4058-bbe1-9c270327db8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6809","nonce":1036888180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6811","nonce":1036888180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6815","nonce":1036888180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6813","nonce":1036888180}]},"public_addr":"172.21.15.189:6809/1036888180","cluster_addr":"172.21.15.189:6811/1036888180","heartbeat_back_addr":"172.21.15.189:6815/1036888180","heartbeat_front_addr":"172.21.15.189:6813/1036888180","state":["exists","up"]},{"osd":5,"uuid":"86c48303-929d-42f4-bf48-cbb32649f95f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6809","nonce":4212601512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6811","nonce":4212601512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6815","nonce":4212601512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6813","nonce":4212601512}]},"public_addr":"172.21.15.203:6809/4212601512","cluster_addr":"172.21.15.203:6811/4212601512","heartbeat_back_addr":"172.21.15.203:6815/4212601512","heartbeat_front_addr":"172.21.15.203:6813/4212601512","state":["exists","up"]},{"osd":6,"uuid":"f0176ed1-c3a7-4d75-b213-fe88fab4441a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6819","nonce":3817684578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6821","nonce":3817684578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6825","nonce":3817684578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6823","nonce":3817684578}]},"public_addr":"172.21.15.159:6819/3817684578","cluster_addr":"172.21.15.159:6821/3817684578","heartbeat_back_addr":"172.21.15.159:6825/3817684578","heartbeat_front_addr":"172.21.15.159:6823/3817684578","state":["exists","up"]},{"osd":7,"uuid":"921bd59d-9415-4559-8e9f-035d336afd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6817","nonce":3936826456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6819","nonce":3936826456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6823","nonce":3936826456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6821","nonce":3936826456}]},"public_addr":"172.21.15.189:6817/3936826456","cluster_addr":"172.21.15.189:6819/3936826456","heartbeat_back_addr":"172.21.15.189:6823/3936826456","heartbeat_front_addr":"172.21.15.189:6821/3936826456","state":["exists","up"]},{"osd":8,"uuid":"ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6817","nonce":4062926645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6819","nonce":4062926645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6823","nonce":4062926645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6821","nonce":4062926645}]},"public_addr":"172.21.15.203:6817/4062926645","cluster_addr":"172.21.15.203:6819/4062926645","heartbeat_back_addr":"172.21.15.203:6823/4062926645","heartbeat_front_addr":"172.21.15.203:6821/4062926645","state":["exists","up"]},{"osd":9,"uuid":"df29a10f-1c60-41cb-9c2d-bc9f1cb174bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6827","nonce":4185848747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6829","nonce":4185848747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6833","nonce":4185848747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6831","nonce":4185848747}]},"public_addr":"172.21.15.159:6827/4185848747","cluster_addr":"172.21.15.159:6829/4185848747","heartbeat_back_addr":"172.21.15.159:6833/4185848747","heartbeat_front_addr":"172.21.15.159:6831/4185848747","state":["exists","up"]},{"osd":10,"uuid":"ccc440b4-4682-4998-bb53-95e24423cd1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6825","nonce":3018377605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6827","nonce":3018377605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6831","nonce":3018377605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6829","nonce":3018377605}]},"public_addr":"172.21.15.189:6825/3018377605","cluster_addr":"172.21.15.189:6827/3018377605","heartbeat_back_addr":"172.21.15.189:6831/3018377605","heartbeat_front_addr":"172.21.15.189:6829/3018377605","state":["exists","up"]},{"osd":11,"uuid":"e84afc5f-8051-4341-8fd0-3afb482ab4df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6825","nonce":2807503891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6827","nonce":2807503891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6831","nonce":2807503891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6829","nonce":2807503891}]},"public_addr":"172.21.15.203:6825/2807503891","cluster_addr":"172.21.15.203:6827/2807503891","heartbeat_back_addr":"172.21.15.203:6831/2807503891","heartbeat_front_addr":"172.21.15.203:6829/2807503891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:00.093677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:25.477205+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:51.345020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:21.077603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:48.822640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:13.516029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:44.382228+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:10.810700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:36.817155+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:07.728553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:34.747540+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:01.249332+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:0/2383222735":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1561371050":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6800/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:6800/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/2108730247":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/3393992615":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1366863724":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/1628344860":"2024-09-20T07:23:03.089190+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:33:37.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.762+0000 7fcff1400640 1 -- 172.21.15.159:0/3150537324 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcfd00a32c0 msgr2=0x7fcfd00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:37.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.762+0000 7fcff1400640 1 --2- 172.21.15.159:0/3150537324 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcfd00a32c0 0x7fcfd00a5780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009230 tx=0x7fcfd80058d0 comp rx=0 tx=0).stop 2024-09-19T07:33:37.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.762+0000 7fcff1400640 1 -- 172.21.15.159:0/3150537324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfec074620 msgr2=0x7fcfec19a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:37.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.762+0000 7fcff1400640 1 --2- 172.21.15.159:0/3150537324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfec074620 0x7fcfec19a510 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc00e9f0 tx=0x7fcfdc00edf0 comp rx=0 tx=0).stop 2024-09-19T07:33:37.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.762+0000 7fcff1400640 1 -- 172.21.15.159:0/3150537324 shutdown_connections 2024-09-19T07:33:37.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.762+0000 7fcff1400640 1 --2- 172.21.15.159:0/3150537324 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcfd00a32c0 0x7fcfd00a5780 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:37.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.762+0000 7fcff1400640 1 --2- 172.21.15.159:0/3150537324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfec10e200 0x7fcfec19ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:37.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.762+0000 7fcff1400640 1 --2- 172.21.15.159:0/3150537324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfec074f40 0x7fcfec19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:37.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.762+0000 7fcff1400640 1 --2- 172.21.15.159:0/3150537324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfec074620 0x7fcfec19a510 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:37.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.762+0000 7fcff1400640 1 -- 172.21.15.159:0/3150537324 >> 172.21.15.159:0/3150537324 conn(0x7fcfec0fd6e0 msgr2=0x7fcfec0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:37.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.763+0000 7fcff1400640 1 -- 172.21.15.159:0/3150537324 shutdown_connections 2024-09-19T07:33:37.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:37.763+0000 7fcff1400640 1 -- 172.21.15.159:0/3150537324 wait complete. 2024-09-19T07:33:38.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:37 smithi159 ceph-mon[30472]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T07:33:38.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:37 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3150537324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:33:38.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:37 smithi203 ceph-mon[35335]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T07:33:38.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:37 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3150537324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:33:38.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:37 smithi189 ceph-mon[34851]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T07:33:38.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:37 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3150537324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:33:38.956 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T07:31:03.518840+0000', 'modified': '2024-09-19T07:31:53.954806+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24542, 'mds_1': 24514, 'mds_2': 24506}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6835/1340279549', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6834', 'nonce': 1340279549}, {'type': 'v1', 'addr': '172.21.15.189:6835', 'nonce': 1340279549}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6835/937970878', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 937970878}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 937970878}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6837/1053443810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 1053443810}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 1053443810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.189:6837/1030825585', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 1030825585}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 1030825585}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.203:6839/3862781462', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6838', 'nonce': 3862781462}, {'type': 'v1', 'addr': '172.21.15.203:6839', 'nonce': 3862781462}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24542': {'gid': 24542, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.189:6839/2361818663', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 2361818663}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 2361818663}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24542, 'qdb_cluster': [24506, 24514, 24542]} 2024-09-19T07:33:38.956 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T07:33:38.956 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T07:33:39.326 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:39.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.898+0000 7fdc98a00640 1 -- 172.21.15.159:0/1165914072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc94109090 msgr2=0x7fdc94110c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:39.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.898+0000 7fdc98a00640 1 --2- 172.21.15.159:0/1165914072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc94109090 0x7fdc94110c10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009be0 tx=0x7fdc8c02a6c0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.898+0000 7fdc98a00640 1 -- 172.21.15.159:0/1165914072 shutdown_connections 2024-09-19T07:33:39.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.898+0000 7fdc98a00640 1 --2- 172.21.15.159:0/1165914072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc94109090 0x7fdc94110c10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.898+0000 7fdc98a00640 1 --2- 172.21.15.159:0/1165914072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc94104800 0x7fdc94108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.898+0000 7fdc98a00640 1 --2- 172.21.15.159:0/1165914072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc94103e50 0x7fdc94104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.898+0000 7fdc98a00640 1 -- 172.21.15.159:0/1165914072 >> 172.21.15.159:0/1165914072 conn(0x7fdc940ff8b0 msgr2=0x7fdc94101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:39.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.898+0000 7fdc98a00640 1 -- 172.21.15.159:0/1165914072 shutdown_connections 2024-09-19T07:33:39.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.898+0000 7fdc98a00640 1 -- 172.21.15.159:0/1165914072 wait complete. 2024-09-19T07:33:39.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.899+0000 7fdc98a00640 1 Processor -- start 2024-09-19T07:33:39.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.899+0000 7fdc98a00640 1 -- start start 2024-09-19T07:33:39.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.900+0000 7fdc98a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc94103e50 0x7fdc9419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.900+0000 7fdc98a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc94104800 0x7fdc9419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.900+0000 7fdc98a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc94109090 0x7fdc9419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.900+0000 7fdc98a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdc941140a0 con 0x7fdc94104800 2024-09-19T07:33:39.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.900+0000 7fdc98a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdc94113f20 con 0x7fdc94103e50 2024-09-19T07:33:39.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.900+0000 7fdc98a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdc94114220 con 0x7fdc94109090 2024-09-19T07:33:39.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.900+0000 7fdc91e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc94104800 0x7fdc9419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.900+0000 7fdc93200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc94109090 0x7fdc9419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.900+0000 7fdc91e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc94104800 0x7fdc9419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37404/0 (socket says 172.21.15.159:37404) 2024-09-19T07:33:39.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.900+0000 7fdc91e00640 1 -- 172.21.15.159:0/2630313711 learned_addr learned my addr 172.21.15.159:0/2630313711 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:39.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.900+0000 7fdc92800640 1 --2- 172.21.15.159:0/2630313711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc94103e50 0x7fdc9419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.900+0000 7fdc91e00640 1 -- 172.21.15.159:0/2630313711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc94109090 msgr2=0x7fdc9419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:39.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.900+0000 7fdc91e00640 1 --2- 172.21.15.159:0/2630313711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc94109090 0x7fdc9419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.900+0000 7fdc91e00640 1 -- 172.21.15.159:0/2630313711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc94103e50 msgr2=0x7fdc9419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:39.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.900+0000 7fdc91e00640 1 --2- 172.21.15.159:0/2630313711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc94103e50 0x7fdc9419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.900+0000 7fdc91e00640 1 -- 172.21.15.159:0/2630313711 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9419f490 con 0x7fdc94104800 2024-09-19T07:33:39.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.901+0000 7fdc91e00640 1 --2- 172.21.15.159:0/2630313711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc94104800 0x7fdc9419aa20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000e9f0 tx=0x7fdc8000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:39.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.901+0000 7fdc8b400640 1 -- 172.21.15.159:0/2630313711 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8000cd80 con 0x7fdc94104800 2024-09-19T07:33:39.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.901+0000 7fdc8b400640 1 -- 172.21.15.159:0/2630313711 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc80005500 con 0x7fdc94104800 2024-09-19T07:33:39.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.901+0000 7fdc8b400640 1 -- 172.21.15.159:0/2630313711 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80010690 con 0x7fdc94104800 2024-09-19T07:33:39.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.901+0000 7fdc98a00640 1 -- 172.21.15.159:0/2630313711 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9419f780 con 0x7fdc94104800 2024-09-19T07:33:39.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.901+0000 7fdc98a00640 1 -- 172.21.15.159:0/2630313711 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc941a7060 con 0x7fdc94104800 2024-09-19T07:33:39.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.905+0000 7fdc9c068740 1 Processor -- start 2024-09-19T07:33:39.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.905+0000 7fdc9c068740 1 -- start start 2024-09-19T07:33:39.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.906+0000 7fdc8b400640 1 -- 172.21.15.159:0/2630313711 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fdc800046c0 con 0x7fdc94104800 2024-09-19T07:33:39.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.906+0000 7fdc9c068740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d264e0 0x5650e7d268c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.906+0000 7fdc9c068740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d26e00 0x5650e7d31180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.906+0000 7fdc9c068740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d316c0 0x5650e7d33ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.906+0000 7fdc9c068740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5650e7c8c0b0 con 0x5650e7d316c0 2024-09-19T07:33:39.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.906+0000 7fdc9c068740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5650e7c33ab0 con 0x5650e7d264e0 2024-09-19T07:33:39.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.906+0000 7fdc9c068740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5650e7c88070 con 0x5650e7d26e00 2024-09-19T07:33:39.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.906+0000 7fdc93200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d264e0 0x5650e7d268c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.906+0000 7fdc91e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d316c0 0x5650e7d33ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.906+0000 7fdc93200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d264e0 0x5650e7d268c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:58236/0 (socket says 172.21.15.159:58236) 2024-09-19T07:33:39.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.906+0000 7fdc93200640 1 -- 172.21.15.159:0/2872345554 learned_addr learned my addr 172.21.15.159:0/2872345554 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:39.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.906+0000 7fdc92800640 1 --2- 172.21.15.159:0/2872345554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d26e00 0x5650e7d31180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.907+0000 7fdc93200640 1 -- 172.21.15.159:0/2872345554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d26e00 msgr2=0x5650e7d31180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:39.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.907+0000 7fdc93200640 1 --2- 172.21.15.159:0/2872345554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d26e00 0x5650e7d31180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.907+0000 7fdc93200640 1 -- 172.21.15.159:0/2872345554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d316c0 msgr2=0x5650e7d33ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:39.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.907+0000 7fdc93200640 1 --2- 172.21.15.159:0/2872345554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d316c0 0x5650e7d33ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.907+0000 7fdc93200640 1 -- 172.21.15.159:0/2872345554 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e7c404a0 con 0x5650e7d264e0 2024-09-19T07:33:39.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.907+0000 7fdc91e00640 1 --2- 172.21.15.159:0/2872345554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d316c0 0x5650e7d33ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:39.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.907+0000 7fdc93200640 1 --2- 172.21.15.159:0/2872345554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d264e0 0x5650e7d268c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009be0 tx=0x7fdc8c005c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65f791aa291bc9fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:39.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.907+0000 7fdc8b400640 1 --2- 172.21.15.159:0/2630313711 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdc780a3240 0x7fdc780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.907+0000 7fdc7f400640 1 -- 172.21.15.159:0/2872345554 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c042030 con 0x5650e7d264e0 2024-09-19T07:33:39.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.907+0000 7fdc7f400640 1 -- 172.21.15.159:0/2872345554 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc8c037040 con 0x5650e7d264e0 2024-09-19T07:33:39.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.907+0000 7fdc7f400640 1 -- 172.21.15.159:0/2872345554 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c036070 con 0x5650e7d264e0 2024-09-19T07:33:39.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.908+0000 7fdc8b400640 1 -- 172.21.15.159:0/2630313711 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fdc800d0060 con 0x7fdc94104800 2024-09-19T07:33:39.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.908+0000 7fdc92800640 1 --2- 172.21.15.159:0/2630313711 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdc780a3240 0x7fdc780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.908+0000 7fdc9c068740 1 -- 172.21.15.159:0/2872345554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d264e0 msgr2=0x5650e7d268c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:39.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.908+0000 7fdc92800640 1 --2- 172.21.15.159:0/2630313711 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdc780a3240 0x7fdc780a5700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x5650e7d238d0 tx=0x7fdc84009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:39.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.908+0000 7fdc9c068740 1 --2- 172.21.15.159:0/2872345554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d264e0 0x5650e7d268c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009be0 tx=0x7fdc8c005c60 comp rx=0 tx=0).stop 2024-09-19T07:33:39.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.909+0000 7fdc9c068740 1 -- 172.21.15.159:0/2872345554 shutdown_connections 2024-09-19T07:33:39.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.909+0000 7fdc9c068740 1 --2- 172.21.15.159:0/2872345554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d316c0 0x5650e7d33ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.909+0000 7fdc9c068740 1 --2- 172.21.15.159:0/2872345554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d26e00 0x5650e7d31180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.909+0000 7fdc9c068740 1 --2- 172.21.15.159:0/2872345554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d264e0 0x5650e7d268c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.909+0000 7fdc9c068740 1 -- 172.21.15.159:0/2872345554 >> 172.21.15.159:0/2872345554 conn(0x5650e7d22a70 msgr2=0x5650e7d22e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:39.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.909+0000 7fdc9c068740 1 -- 172.21.15.159:0/2872345554 shutdown_connections 2024-09-19T07:33:39.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.909+0000 7fdc9c068740 1 -- 172.21.15.159:0/2872345554 wait complete. 2024-09-19T07:33:39.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.909+0000 7fdc9c068740 1 -- start start 2024-09-19T07:33:39.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.909+0000 7fdc9c068740 1 Processor -- start 2024-09-19T07:33:39.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.910+0000 7fdc9c068740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d264e0 0x5650e7d2e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.910+0000 7fdc9c068740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d26e00 0x5650e7d2eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.910+0000 7fdc9c068740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d316c0 0x5650e7d2f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.910+0000 7fdc9c068740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5650e7d34330 con 0x5650e7d316c0 2024-09-19T07:33:39.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.910+0000 7fdc9c068740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5650e7c14920 con 0x5650e7d26e00 2024-09-19T07:33:39.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.910+0000 7fdc9c068740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5650e7d344b0 con 0x5650e7d264e0 2024-09-19T07:33:39.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.910+0000 7fdc9c068740 10 client.0 _ll_register_callbacks cb 0x5650e7c388a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:33:39.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.910+0000 7fdc91e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d26e00 0x5650e7d2eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.910+0000 7fdc91e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d26e00 0x5650e7d2eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:58252/0 (socket says 172.21.15.159:58252) 2024-09-19T07:33:39.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.910+0000 7fdc91e00640 1 -- 172.21.15.159:0/444887102 learned_addr learned my addr 172.21.15.159:0/444887102 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:39.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.910+0000 7fdc92800640 1 --2- 172.21.15.159:0/444887102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d264e0 0x5650e7d2e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.910+0000 7fdc93200640 1 --2- 172.21.15.159:0/444887102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d316c0 0x5650e7d2f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.911+0000 7fdc91e00640 1 -- 172.21.15.159:0/444887102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d264e0 msgr2=0x5650e7d2e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:39.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.911+0000 7fdc91e00640 1 --2- 172.21.15.159:0/444887102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d264e0 0x5650e7d2e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.911+0000 7fdc91e00640 1 -- 172.21.15.159:0/444887102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d316c0 msgr2=0x5650e7d2f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:39.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.911+0000 7fdc91e00640 1 --2- 172.21.15.159:0/444887102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d316c0 0x5650e7d2f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.911+0000 7fdc91e00640 1 -- 172.21.15.159:0/444887102 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e79fd2b0 con 0x5650e7d26e00 2024-09-19T07:33:39.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.911+0000 7fdc92800640 1 --2- 172.21.15.159:0/444887102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d264e0 0x5650e7d2e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:39.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.911+0000 7fdc91e00640 1 --2- 172.21.15.159:0/444887102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d26e00 0x5650e7d2eae0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000e9c0 tx=0x7fdc80009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f33f9c4b5d619bf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:39.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.911+0000 7fdc73e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-19T07:33:39.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.911+0000 7fdc73e00640 1 -- 172.21.15.159:0/444887102 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8009c0a0 con 0x5650e7d26e00 2024-09-19T07:33:39.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.911+0000 7fdc73e00640 1 -- 172.21.15.159:0/444887102 <== mon.2 v2:172.21.15.189:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fdc800a4ab0 con 0x5650e7d26e00 2024-09-19T07:33:39.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.911+0000 7fdc9c068740 1 -- 172.21.15.159:0/444887102 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc8c037040 con 0x5650e7d26e00 2024-09-19T07:33:39.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.912+0000 7fdc73e00640 1 -- 172.21.15.159:0/444887102 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdc8009bc30 con 0x5650e7d26e00 2024-09-19T07:33:39.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.912+0000 7fdc9c068740 10 client.25088 fetch_fsmap learned FSMap version 25 2024-09-19T07:33:39.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.912+0000 7fdc9c068740 1 -- 172.21.15.159:0/444887102 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5650e7d488a0 con 0x5650e7d26e00 2024-09-19T07:33:39.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.912+0000 7fdc73e00640 1 -- 172.21.15.159:0/444887102 <== mon.2 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc8c037040 con 0x5650e7d26e00 2024-09-19T07:33:39.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.912+0000 7fdc73e00640 1 -- 172.21.15.159:0/444887102 <== mon.2 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdc800d20d0 con 0x5650e7d26e00 2024-09-19T07:33:39.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.912+0000 7fdc9c068740 10 client.25088 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T07:33:39.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.912+0000 7fdc9c068740 10 client.25088 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T07:33:39.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.912+0000 7fdc9c068740 20 client.25088 populate_metadata read hostname 'smithi159' 2024-09-19T07:33:39.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.912+0000 7fdc9c068740 1 --2- 172.21.15.159:0/444887102 >> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] conn(0x5650e7d4aec0 0x5650e7d4c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.912+0000 7fdc9c068740 4 client.25088 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:33:39.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.912+0000 7fdc9c068740 1 -- 172.21.15.159:0/444887102 --> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650e7c99620 con 0x5650e7d4aec0 2024-09-19T07:33:39.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.913+0000 7fdc92800640 1 --2- 172.21.15.159:0/444887102 >> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] conn(0x5650e7d4aec0 0x5650e7d4c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.913+0000 7fdc92800640 1 --2- 172.21.15.159:0/444887102 >> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] conn(0x5650e7d4aec0 0x5650e7d4c330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=93babadb419b3c6 server_cookie=8bb89e8adc81500b in_seq=0 out_seq=0 2024-09-19T07:33:39.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.913+0000 7fdc73e00640 10 client.25088 ms_handle_connect on v2:172.21.15.189:6836/1030825585 2024-09-19T07:33:39.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.914+0000 7fdc73e00640 1 -- 172.21.15.159:0/444887102 <== mds.2 v2:172.21.15.189:6836/1030825585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650e7c99620 con 0x5650e7d4aec0 2024-09-19T07:33:39.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.914+0000 7fdc73e00640 10 client.25088 handle_command_reply: tid=0 2024-09-19T07:33:39.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.914+0000 7fdc9c068740 1 client.25088 shutdown 2024-09-19T07:33:39.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.914+0000 7fdc9c068740 1 -- 172.21.15.159:0/444887102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d26e00 msgr2=0x5650e7d2eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:39.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.914+0000 7fdc9c068740 1 --2- 172.21.15.159:0/444887102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d26e00 0x5650e7d2eae0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000e9c0 tx=0x7fdc80009310 comp rx=0 tx=0).stop 2024-09-19T07:33:39.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.914+0000 7fdc9c068740 1 -- 172.21.15.159:0/444887102 shutdown_connections 2024-09-19T07:33:39.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.914+0000 7fdc9c068740 1 --2- 172.21.15.159:0/444887102 >> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] conn(0x5650e7d4aec0 0x5650e7d4c330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.914+0000 7fdc9c068740 1 --2- 172.21.15.159:0/444887102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d316c0 0x5650e7d2f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.914+0000 7fdc9c068740 1 --2- 172.21.15.159:0/444887102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d26e00 0x5650e7d2eae0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.914+0000 7fdc9c068740 1 --2- 172.21.15.159:0/444887102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d264e0 0x5650e7d2e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.914+0000 7fdc73e00640 0 client.25088 ms_handle_reset on v2:172.21.15.189:6836/1030825585 2024-09-19T07:33:39.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.914+0000 7fdc9c068740 1 -- 172.21.15.159:0/444887102 >> 172.21.15.159:0/444887102 conn(0x5650e7d22a70 msgr2=0x5650e7d28750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:39.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.914+0000 7fdc9c068740 1 -- 172.21.15.159:0/444887102 shutdown_connections 2024-09-19T07:33:39.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.914+0000 7fdc9c068740 1 -- 172.21.15.159:0/444887102 wait complete. 2024-09-19T07:33:39.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.914+0000 7fdc9c068740 20 client.25088 trim_cache size 0 max 16384 2024-09-19T07:33:39.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.932+0000 7fdc9c068740 1 Processor -- start 2024-09-19T07:33:39.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.932+0000 7fdc9c068740 1 -- start start 2024-09-19T07:33:39.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.933+0000 7fdc9c068740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d264e0 0x5650e7d28f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.933+0000 7fdc9c068740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d26e00 0x5650e7d2ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.933+0000 7fdc9c068740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d316c0 0x5650e7d2d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.933+0000 7fdc9c068740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5650e7d4a670 con 0x5650e7d264e0 2024-09-19T07:33:39.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.933+0000 7fdc9c068740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5650e7d4ad40 con 0x5650e7d316c0 2024-09-19T07:33:39.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.933+0000 7fdc9c068740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5650e7d4b410 con 0x5650e7d26e00 2024-09-19T07:33:39.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.933+0000 7fdc92800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d264e0 0x5650e7d28f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.933+0000 7fdc92800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d264e0 0x5650e7d28f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37424/0 (socket says 172.21.15.159:37424) 2024-09-19T07:33:39.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.933+0000 7fdc92800640 1 -- 172.21.15.159:0/2773516555 learned_addr learned my addr 172.21.15.159:0/2773516555 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:39.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.933+0000 7fdc93200640 1 --2- 172.21.15.159:0/2773516555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d316c0 0x5650e7d2d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.933+0000 7fdc91e00640 1 --2- 172.21.15.159:0/2773516555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d26e00 0x5650e7d2ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.933+0000 7fdc93200640 1 -- 172.21.15.159:0/2773516555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d26e00 msgr2=0x5650e7d2ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:39.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.933+0000 7fdc93200640 1 --2- 172.21.15.159:0/2773516555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d26e00 0x5650e7d2ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.933+0000 7fdc93200640 1 -- 172.21.15.159:0/2773516555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d264e0 msgr2=0x5650e7d28f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:39.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.933+0000 7fdc93200640 1 --2- 172.21.15.159:0/2773516555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d264e0 0x5650e7d28f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.933+0000 7fdc93200640 1 -- 172.21.15.159:0/2773516555 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e7d26310 con 0x5650e7d316c0 2024-09-19T07:33:39.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.933+0000 7fdc92800640 1 --2- 172.21.15.159:0/2773516555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d264e0 0x5650e7d28f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:39.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.934+0000 7fdc93200640 1 --2- 172.21.15.159:0/2773516555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d316c0 0x5650e7d2d3b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c02abe0 tx=0x7fdc8c02a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e28b5b23cbef3d46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:39.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.934+0000 7fdc7ea00640 1 -- 172.21.15.159:0/2773516555 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c042040 con 0x5650e7d316c0 2024-09-19T07:33:39.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.934+0000 7fdc7ea00640 1 -- 172.21.15.159:0/2773516555 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc8c04b070 con 0x5650e7d316c0 2024-09-19T07:33:39.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.934+0000 7fdc7ea00640 1 -- 172.21.15.159:0/2773516555 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c036040 con 0x5650e7d316c0 2024-09-19T07:33:39.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.935+0000 7fdc9c068740 1 -- 172.21.15.159:0/2773516555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d316c0 msgr2=0x5650e7d2d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:39.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.935+0000 7fdc9c068740 1 --2- 172.21.15.159:0/2773516555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d316c0 0x5650e7d2d3b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c02abe0 tx=0x7fdc8c02a6c0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.935+0000 7fdc9c068740 1 -- 172.21.15.159:0/2773516555 shutdown_connections 2024-09-19T07:33:39.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.935+0000 7fdc9c068740 1 --2- 172.21.15.159:0/2773516555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d316c0 0x5650e7d2d3b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.935+0000 7fdc9c068740 1 --2- 172.21.15.159:0/2773516555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d26e00 0x5650e7d2ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.935+0000 7fdc9c068740 1 --2- 172.21.15.159:0/2773516555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d264e0 0x5650e7d28f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.935+0000 7fdc9c068740 1 -- 172.21.15.159:0/2773516555 >> 172.21.15.159:0/2773516555 conn(0x5650e7d22a70 msgr2=0x5650e7d22e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:39.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.935+0000 7fdc9c068740 1 -- 172.21.15.159:0/2773516555 shutdown_connections 2024-09-19T07:33:39.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.935+0000 7fdc9c068740 1 -- 172.21.15.159:0/2773516555 wait complete. 2024-09-19T07:33:39.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.935+0000 7fdc9c068740 1 -- start start 2024-09-19T07:33:39.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.936+0000 7fdc9c068740 1 Processor -- start 2024-09-19T07:33:39.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.936+0000 7fdc9c068740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d264e0 0x5650e7d54710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.936+0000 7fdc9c068740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d26e00 0x5650e7d54c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.936+0000 7fdc9c068740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d316c0 0x5650e7d4db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.936+0000 7fdc9c068740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5650e7d4c1b0 con 0x5650e7d264e0 2024-09-19T07:33:39.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.936+0000 7fdc9c068740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5650e7d4bae0 con 0x5650e7d26e00 2024-09-19T07:33:39.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.936+0000 7fdc9c068740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5650e7d4c880 con 0x5650e7d316c0 2024-09-19T07:33:39.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.936+0000 7fdc9c068740 10 client.0 _ll_register_callbacks cb 0x7fdc800d3830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:33:39.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.936+0000 7fdc92800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d264e0 0x5650e7d54710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.936+0000 7fdc91e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d26e00 0x5650e7d54c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.936+0000 7fdc92800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d264e0 0x5650e7d54710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37432/0 (socket says 172.21.15.159:37432) 2024-09-19T07:33:39.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.936+0000 7fdc92800640 1 -- 172.21.15.159:0/3527935432 learned_addr learned my addr 172.21.15.159:0/3527935432 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:39.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.936+0000 7fdc93200640 1 --2- 172.21.15.159:0/3527935432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d316c0 0x5650e7d4db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.937+0000 7fdc91e00640 1 -- 172.21.15.159:0/3527935432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d316c0 msgr2=0x5650e7d4db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:39.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.937+0000 7fdc91e00640 1 --2- 172.21.15.159:0/3527935432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d316c0 0x5650e7d4db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.937+0000 7fdc91e00640 1 -- 172.21.15.159:0/3527935432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d264e0 msgr2=0x5650e7d54710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:39.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.937+0000 7fdc91e00640 1 --2- 172.21.15.159:0/3527935432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d264e0 0x5650e7d54710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.937+0000 7fdc91e00640 1 -- 172.21.15.159:0/3527935432 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e7d46530 con 0x5650e7d26e00 2024-09-19T07:33:39.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.937+0000 7fdc93200640 1 --2- 172.21.15.159:0/3527935432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d316c0 0x5650e7d4db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:39.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.937+0000 7fdc92800640 1 --2- 172.21.15.159:0/3527935432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d264e0 0x5650e7d54710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:39.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.937+0000 7fdc91e00640 1 --2- 172.21.15.159:0/3527935432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d26e00 0x5650e7d54c50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdc80098730 tx=0x7fdc8009c0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1378a9cd5fc59387 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:39.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.937+0000 7fdc73e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-19T07:33:39.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.937+0000 7fdc73e00640 1 -- 172.21.15.159:0/3527935432 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8009be10 con 0x5650e7d26e00 2024-09-19T07:33:39.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.937+0000 7fdc9c068740 1 -- 172.21.15.159:0/3527935432 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc8c04b070 con 0x5650e7d26e00 2024-09-19T07:33:39.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.937+0000 7fdc73e00640 1 -- 172.21.15.159:0/3527935432 <== mon.2 v2:172.21.15.189:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fdc800a4cb0 con 0x5650e7d26e00 2024-09-19T07:33:39.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.938+0000 7fdc73e00640 1 -- 172.21.15.159:0/3527935432 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdc8009c9d0 con 0x5650e7d26e00 2024-09-19T07:33:39.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.938+0000 7fdc9c068740 10 client.25094 fetch_fsmap learned FSMap version 25 2024-09-19T07:33:39.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.938+0000 7fdc9c068740 1 -- 172.21.15.159:0/3527935432 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5650e7d52b20 con 0x5650e7d26e00 2024-09-19T07:33:39.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.938+0000 7fdc73e00640 1 -- 172.21.15.159:0/3527935432 <== mon.2 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc8c04b070 con 0x5650e7d26e00 2024-09-19T07:33:39.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.938+0000 7fdc73e00640 1 -- 172.21.15.159:0/3527935432 <== mon.2 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdc800a0160 con 0x5650e7d26e00 2024-09-19T07:33:39.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.938+0000 7fdc9c068740 10 client.25094 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T07:33:39.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.938+0000 7fdc9c068740 10 client.25094 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T07:33:39.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.938+0000 7fdc9c068740 20 client.25094 populate_metadata read hostname 'smithi159' 2024-09-19T07:33:39.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.938+0000 7fdc9c068740 1 --2- 172.21.15.159:0/3527935432 >> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] conn(0x5650e7db9810 0x5650e7db8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:39.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.938+0000 7fdc9c068740 4 client.25094 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:33:39.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.938+0000 7fdc9c068740 1 -- 172.21.15.159:0/3527935432 --> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] -- command(tid 0: {"prefix": "status"}) -- 0x5650e7c93b30 con 0x5650e7db9810 2024-09-19T07:33:39.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.938+0000 7fdc92800640 1 --2- 172.21.15.159:0/3527935432 >> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] conn(0x5650e7db9810 0x5650e7db8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:39.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.939+0000 7fdc92800640 1 --2- 172.21.15.159:0/3527935432 >> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] conn(0x5650e7db9810 0x5650e7db8520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f928290f92b4922 server_cookie=ab769a50c14289d6 in_seq=0 out_seq=0 2024-09-19T07:33:39.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.939+0000 7fdc73e00640 10 client.25094 ms_handle_connect on v2:172.21.15.189:6836/1030825585 2024-09-19T07:33:39.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.939+0000 7fdc73e00640 1 -- 172.21.15.159:0/3527935432 <== mds.2 v2:172.21.15.189:6836/1030825585 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5650e7c93b30 con 0x5650e7db9810 2024-09-19T07:33:39.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.939+0000 7fdc73e00640 10 client.25094 handle_command_reply: tid=0 2024-09-19T07:33:39.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.939+0000 7fdc9c068740 1 client.25094 shutdown 2024-09-19T07:33:39.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.939+0000 7fdc9c068740 1 -- 172.21.15.159:0/3527935432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d26e00 msgr2=0x5650e7d54c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:39.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.939+0000 7fdc9c068740 1 --2- 172.21.15.159:0/3527935432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d26e00 0x5650e7d54c50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdc80098730 tx=0x7fdc8009c0a0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.939+0000 7fdc9c068740 1 -- 172.21.15.159:0/3527935432 shutdown_connections 2024-09-19T07:33:39.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.939+0000 7fdc9c068740 1 --2- 172.21.15.159:0/3527935432 >> [v2:172.21.15.189:6836/1030825585,v1:172.21.15.189:6837/1030825585] conn(0x5650e7db9810 0x5650e7db8520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.939+0000 7fdc9c068740 1 --2- 172.21.15.159:0/3527935432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5650e7d316c0 0x5650e7d4db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.939+0000 7fdc9c068740 1 --2- 172.21.15.159:0/3527935432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650e7d26e00 0x5650e7d54c50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.939+0000 7fdc73e00640 0 client.25094 ms_handle_reset on v2:172.21.15.189:6836/1030825585 2024-09-19T07:33:39.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.939+0000 7fdc9c068740 1 --2- 172.21.15.159:0/3527935432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650e7d264e0 0x5650e7d54710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.939+0000 7fdc9c068740 1 -- 172.21.15.159:0/3527935432 >> 172.21.15.159:0/3527935432 conn(0x5650e7d22a70 msgr2=0x5650e7d2cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:39.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.940+0000 7fdc9c068740 1 -- 172.21.15.159:0/3527935432 shutdown_connections 2024-09-19T07:33:39.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.940+0000 7fdc9c068740 1 -- 172.21.15.159:0/3527935432 wait complete. 2024-09-19T07:33:39.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.940+0000 7fdc9c068740 20 client.25094 trim_cache size 0 max 16384 2024-09-19T07:33:39.950 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-19T07:33:39.950 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "c9d18048-7657-11ef-bceb-c7b262605968", 2024-09-19T07:33:39.950 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 2, 2024-09-19T07:33:39.950 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 24506, 2024-09-19T07:33:39.950 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-09-19T07:33:39.951 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-09-19T07:33:39.951 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-09-19T07:33:39.951 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 109.053484482, 2024-09-19T07:33:39.951 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 25, 2024-09-19T07:33:39.951 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 100, 2024-09-19T07:33:39.951 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 98, 2024-09-19T07:33:39.952 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 184.06947391099999 2024-09-19T07:33:39.952 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-19T07:33:39.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.940+0000 7fdc98a00640 1 -- 172.21.15.159:0/2630313711 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdc780a3240 msgr2=0x7fdc780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:39.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.940+0000 7fdc98a00640 1 --2- 172.21.15.159:0/2630313711 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdc780a3240 0x7fdc780a5700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x5650e7d238d0 tx=0x7fdc84009260 comp rx=0 tx=0).stop 2024-09-19T07:33:39.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.940+0000 7fdc98a00640 1 -- 172.21.15.159:0/2630313711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc94104800 msgr2=0x7fdc9419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:39.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.940+0000 7fdc98a00640 1 --2- 172.21.15.159:0/2630313711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc94104800 0x7fdc9419aa20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000e9f0 tx=0x7fdc8000edf0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.940+0000 7fdc98a00640 1 -- 172.21.15.159:0/2630313711 shutdown_connections 2024-09-19T07:33:39.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.940+0000 7fdc98a00640 1 --2- 172.21.15.159:0/2630313711 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fdc780a3240 0x7fdc780a5700 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.940+0000 7fdc98a00640 1 --2- 172.21.15.159:0/2630313711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc94109090 0x7fdc9419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.940+0000 7fdc98a00640 1 --2- 172.21.15.159:0/2630313711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc94104800 0x7fdc9419aa20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.940+0000 7fdc98a00640 1 --2- 172.21.15.159:0/2630313711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc94103e50 0x7fdc9419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:39.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.940+0000 7fdc98a00640 1 -- 172.21.15.159:0/2630313711 >> 172.21.15.159:0/2630313711 conn(0x7fdc940ff8b0 msgr2=0x7fdc94100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:39.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.941+0000 7fdc98a00640 1 -- 172.21.15.159:0/2630313711 shutdown_connections 2024-09-19T07:33:39.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:39.941+0000 7fdc98a00640 1 -- 172.21.15.159:0/2630313711 wait complete. 2024-09-19T07:33:40.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:39 smithi159 ceph-mon[30472]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T07:33:40.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:39 smithi203 ceph-mon[35335]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T07:33:40.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:39 smithi189 ceph-mon[34851]: pgmap v394: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T07:33:40.845 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T07:33:41.218 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:41.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.789+0000 7fa4a1c00640 1 -- 172.21.15.159:0/3543073997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa49c103ff0 msgr2=0x7fa49c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:41.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.789+0000 7fa4a1c00640 1 --2- 172.21.15.159:0/3543073997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa49c103ff0 0x7fa49c1043d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009900 tx=0x7fa48c02a3f0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.789+0000 7fa4a1c00640 1 -- 172.21.15.159:0/3543073997 shutdown_connections 2024-09-19T07:33:41.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.789+0000 7fa4a1c00640 1 --2- 172.21.15.159:0/3543073997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa49c109120 0x7fa49c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.789+0000 7fa4a1c00640 1 --2- 172.21.15.159:0/3543073997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa49c1049a0 0x7fa49c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.789+0000 7fa4a1c00640 1 --2- 172.21.15.159:0/3543073997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa49c103ff0 0x7fa49c1043d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.789+0000 7fa4a1c00640 1 -- 172.21.15.159:0/3543073997 >> 172.21.15.159:0/3543073997 conn(0x7fa49c0ffa90 msgr2=0x7fa49c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:41.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.789+0000 7fa4a1c00640 1 -- 172.21.15.159:0/3543073997 shutdown_connections 2024-09-19T07:33:41.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.789+0000 7fa4a1c00640 1 -- 172.21.15.159:0/3543073997 wait complete. 2024-09-19T07:33:41.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.790+0000 7fa4a1c00640 1 Processor -- start 2024-09-19T07:33:41.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.790+0000 7fa4a1c00640 1 -- start start 2024-09-19T07:33:41.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.790+0000 7fa4a1c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa49c103ff0 0x7fa49c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.790+0000 7fa4a1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa49c1049a0 0x7fa49c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.790+0000 7fa4a1c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa49c109120 0x7fa49c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.790+0000 7fa4a1c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c114170 con 0x7fa49c1049a0 2024-09-19T07:33:41.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.790+0000 7fa4a1c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c113ff0 con 0x7fa49c109120 2024-09-19T07:33:41.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.790+0000 7fa4a1c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c1142f0 con 0x7fa49c103ff0 2024-09-19T07:33:41.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.791+0000 7fa492000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa49c1049a0 0x7fa49c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.791+0000 7fa49b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa49c109120 0x7fa49c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.791+0000 7fa49b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa49c109120 0x7fa49c19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:58288/0 (socket says 172.21.15.159:58288) 2024-09-19T07:33:41.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.791+0000 7fa49b400640 1 -- 172.21.15.159:0/3811336671 learned_addr learned my addr 172.21.15.159:0/3811336671 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:41.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.791+0000 7fa492000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa49c1049a0 0x7fa49c19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37438/0 (socket says 172.21.15.159:37438) 2024-09-19T07:33:41.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.791+0000 7fa49aa00640 1 --2- 172.21.15.159:0/3811336671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa49c103ff0 0x7fa49c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.791+0000 7fa492000640 1 -- 172.21.15.159:0/3811336671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa49c103ff0 msgr2=0x7fa49c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:41.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.791+0000 7fa492000640 1 --2- 172.21.15.159:0/3811336671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa49c103ff0 0x7fa49c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.791+0000 7fa492000640 1 -- 172.21.15.159:0/3811336671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa49c109120 msgr2=0x7fa49c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:41.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.791+0000 7fa492000640 1 --2- 172.21.15.159:0/3811336671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa49c109120 0x7fa49c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.791+0000 7fa492000640 1 -- 172.21.15.159:0/3811336671 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49c19f5b0 con 0x7fa49c1049a0 2024-09-19T07:33:41.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.791+0000 7fa49b400640 1 --2- 172.21.15.159:0/3811336671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa49c109120 0x7fa49c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:41.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.791+0000 7fa492000640 1 --2- 172.21.15.159:0/3811336671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa49c1049a0 0x7fa49c19ab40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa484005fd0 tx=0x7fa48400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:41.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.792+0000 7fa493e00640 1 -- 172.21.15.159:0/3811336671 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484009c30 con 0x7fa49c1049a0 2024-09-19T07:33:41.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.792+0000 7fa493e00640 1 -- 172.21.15.159:0/3811336671 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa484009dd0 con 0x7fa49c1049a0 2024-09-19T07:33:41.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.792+0000 7fa493e00640 1 -- 172.21.15.159:0/3811336671 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484010460 con 0x7fa49c1049a0 2024-09-19T07:33:41.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.792+0000 7fa4a1c00640 1 -- 172.21.15.159:0/3811336671 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49c19f8a0 con 0x7fa49c1049a0 2024-09-19T07:33:41.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.792+0000 7fa4a1c00640 1 -- 172.21.15.159:0/3811336671 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa49c1a7180 con 0x7fa49c1049a0 2024-09-19T07:33:41.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.795+0000 7fa4a4664740 1 Processor -- start 2024-09-19T07:33:41.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.795+0000 7fa4a4664740 1 -- start start 2024-09-19T07:33:41.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.795+0000 7fa493e00640 1 -- 172.21.15.159:0/3811336671 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fa484003f50 con 0x7fa49c1049a0 2024-09-19T07:33:41.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.796+0000 7fa4a4664740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 0x562d917c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.796+0000 7fa4a4664740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 0x562d917cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.796+0000 7fa4a4664740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917ce510 0x562d917d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.796+0000 7fa4a4664740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562d917290b0 con 0x562d917c3330 2024-09-19T07:33:41.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.796+0000 7fa4a4664740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562d916d0ab0 con 0x562d917c3c50 2024-09-19T07:33:41.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.796+0000 7fa4a4664740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562d91725070 con 0x562d917ce510 2024-09-19T07:33:41.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.796+0000 7fa49b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 0x562d917c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.796+0000 7fa492000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917ce510 0x562d917d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.796+0000 7fa49b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 0x562d917c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37442/0 (socket says 172.21.15.159:37442) 2024-09-19T07:33:41.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.796+0000 7fa49b400640 1 -- 172.21.15.159:0/2560340729 learned_addr learned my addr 172.21.15.159:0/2560340729 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:41.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.796+0000 7fa49aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 0x562d917cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.796+0000 7fa492000640 1 -- 172.21.15.159:0/2560340729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 msgr2=0x562d917cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:41.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.796+0000 7fa492000640 1 --2- 172.21.15.159:0/2560340729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 0x562d917cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.797+0000 7fa492000640 1 -- 172.21.15.159:0/2560340729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 msgr2=0x562d917c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:41.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.797+0000 7fa492000640 1 --2- 172.21.15.159:0/2560340729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 0x562d917c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.797+0000 7fa492000640 1 -- 172.21.15.159:0/2560340729 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d916dd4a0 con 0x562d917ce510 2024-09-19T07:33:41.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.797+0000 7fa492000640 1 --2- 172.21.15.159:0/2560340729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917ce510 0x562d917d0900 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa4840cd1c0 tx=0x7fa4840d3c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b6ed5a92561ea7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:41.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.797+0000 7fa493e00640 1 --2- 172.21.15.159:0/3811336671 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa4800a32c0 0x7fa4800a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.797+0000 7fa48be00640 1 -- 172.21.15.159:0/2560340729 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4840d8b70 con 0x562d917ce510 2024-09-19T07:33:41.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.797+0000 7fa48be00640 1 -- 172.21.15.159:0/2560340729 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48409bdf0 con 0x562d917ce510 2024-09-19T07:33:41.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.797+0000 7fa48be00640 1 -- 172.21.15.159:0/2560340729 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4840ba9b0 con 0x562d917ce510 2024-09-19T07:33:41.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.798+0000 7fa493e00640 1 -- 172.21.15.159:0/3811336671 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa4840cfb30 con 0x7fa49c1049a0 2024-09-19T07:33:41.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.798+0000 7fa49b400640 1 --2- 172.21.15.159:0/3811336671 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa4800a32c0 0x7fa4800a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.798+0000 7fa4a4664740 1 -- 172.21.15.159:0/2560340729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917ce510 msgr2=0x562d917d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:41.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.798+0000 7fa4a4664740 1 --2- 172.21.15.159:0/2560340729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917ce510 0x562d917d0900 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa4840cd1c0 tx=0x7fa4840d3c30 comp rx=0 tx=0).stop 2024-09-19T07:33:41.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.798+0000 7fa49b400640 1 --2- 172.21.15.159:0/3811336671 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa4800a32c0 0x7fa4800a5780 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa4940116f0 tx=0x7fa494009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:41.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.798+0000 7fa4a4664740 1 -- 172.21.15.159:0/2560340729 shutdown_connections 2024-09-19T07:33:41.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.798+0000 7fa4a4664740 1 --2- 172.21.15.159:0/2560340729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917ce510 0x562d917d0900 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.798+0000 7fa4a4664740 1 --2- 172.21.15.159:0/2560340729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 0x562d917cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.798+0000 7fa4a4664740 1 --2- 172.21.15.159:0/2560340729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 0x562d917c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.798+0000 7fa4a4664740 1 -- 172.21.15.159:0/2560340729 >> 172.21.15.159:0/2560340729 conn(0x562d917bf8c0 msgr2=0x562d917bfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:41.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.798+0000 7fa4a4664740 1 -- 172.21.15.159:0/2560340729 shutdown_connections 2024-09-19T07:33:41.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.799+0000 7fa4a4664740 1 -- 172.21.15.159:0/2560340729 wait complete. 2024-09-19T07:33:41.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.799+0000 7fa4a4664740 1 -- start start 2024-09-19T07:33:41.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.799+0000 7fa4a4664740 1 Processor -- start 2024-09-19T07:33:41.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.800+0000 7fa4a4664740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 0x562d917cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.800+0000 7fa4a4664740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 0x562d917cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.800+0000 7fa4a4664740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917ce510 0x562d917cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.800+0000 7fa4a4664740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562d917d1060 con 0x562d917c3330 2024-09-19T07:33:41.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.800+0000 7fa4a4664740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562d916b1920 con 0x562d917c3c50 2024-09-19T07:33:41.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.800+0000 7fa4a4664740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562d917d11e0 con 0x562d917ce510 2024-09-19T07:33:41.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.800+0000 7fa4a4664740 10 client.0 _ll_register_callbacks cb 0x562d916d58a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:33:41.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.800+0000 7fa49aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 0x562d917cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.800+0000 7fa492000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 0x562d917cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.800+0000 7fa49b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917ce510 0x562d917cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.800+0000 7fa49aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 0x562d917cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37446/0 (socket says 172.21.15.159:37446) 2024-09-19T07:33:41.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.800+0000 7fa492000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 0x562d917cba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:58314/0 (socket says 172.21.15.159:58314) 2024-09-19T07:33:41.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.800+0000 7fa49aa00640 1 -- 172.21.15.159:0/4147820268 learned_addr learned my addr 172.21.15.159:0/4147820268 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:41.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.800+0000 7fa49aa00640 1 -- 172.21.15.159:0/4147820268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917ce510 msgr2=0x562d917cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:41.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.800+0000 7fa49aa00640 1 --2- 172.21.15.159:0/4147820268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917ce510 0x562d917cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.800+0000 7fa49aa00640 1 -- 172.21.15.159:0/4147820268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 msgr2=0x562d917cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:41.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.800+0000 7fa49aa00640 1 --2- 172.21.15.159:0/4147820268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 0x562d917cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.801+0000 7fa49aa00640 1 -- 172.21.15.159:0/4147820268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d9149a2b0 con 0x562d917c3330 2024-09-19T07:33:41.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.801+0000 7fa492000640 1 --2- 172.21.15.159:0/4147820268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 0x562d917cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:41.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.801+0000 7fa49aa00640 1 --2- 172.21.15.159:0/4147820268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 0x562d917cb4f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009900 tx=0x7fa48c004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef2f8348741af61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:41.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.801+0000 7fa488c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-19T07:33:41.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.801+0000 7fa488c00640 1 -- 172.21.15.159:0/4147820268 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c035040 con 0x562d917c3330 2024-09-19T07:33:41.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.801+0000 7fa4a4664740 1 -- 172.21.15.159:0/4147820268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa48409bdf0 con 0x562d917c3330 2024-09-19T07:33:41.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.801+0000 7fa488c00640 1 -- 172.21.15.159:0/4147820268 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa48c005450 con 0x562d917c3330 2024-09-19T07:33:41.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.801+0000 7fa488c00640 1 -- 172.21.15.159:0/4147820268 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa48c04f050 con 0x562d917c3330 2024-09-19T07:33:41.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.801+0000 7fa4a4664740 10 client.15075 fetch_fsmap learned FSMap version 25 2024-09-19T07:33:41.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.801+0000 7fa4a4664740 1 -- 172.21.15.159:0/4147820268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d917dcdb0 con 0x562d917c3330 2024-09-19T07:33:41.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.802+0000 7fa488c00640 1 -- 172.21.15.159:0/4147820268 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48409bdf0 con 0x562d917c3330 2024-09-19T07:33:41.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.802+0000 7fa488c00640 1 -- 172.21.15.159:0/4147820268 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa48c003a60 con 0x562d917c3330 2024-09-19T07:33:41.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.802+0000 7fa4a4664740 10 client.15075 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T07:33:41.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.802+0000 7fa4a4664740 10 client.15075 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T07:33:41.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.802+0000 7fa4a4664740 20 client.15075 populate_metadata read hostname 'smithi159' 2024-09-19T07:33:41.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.802+0000 7fa4a4664740 1 --2- 172.21.15.159:0/4147820268 >> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] conn(0x562d917df3e0 0x562d917e0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.802+0000 7fa4a4664740 4 client.15075 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:33:41.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.802+0000 7fa4a4664740 1 -- 172.21.15.159:0/4147820268 --> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d91736620 con 0x562d917df3e0 2024-09-19T07:33:41.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.803+0000 7fa492000640 1 --2- 172.21.15.159:0/4147820268 >> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] conn(0x562d917df3e0 0x562d917e0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.803+0000 7fa492000640 1 --2- 172.21.15.159:0/4147820268 >> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] conn(0x562d917df3e0 0x562d917e0850 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60a2d2c0c0eb1883 server_cookie=9206625b7a2fa247 in_seq=0 out_seq=0 2024-09-19T07:33:41.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.803+0000 7fa488c00640 10 client.15075 ms_handle_connect on v2:172.21.15.203:6838/3862781462 2024-09-19T07:33:41.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.804+0000 7fa488c00640 1 -- 172.21.15.159:0/4147820268 <== mds.1 v2:172.21.15.203:6838/3862781462 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d91736620 con 0x562d917df3e0 2024-09-19T07:33:41.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.804+0000 7fa488c00640 10 client.15075 handle_command_reply: tid=0 2024-09-19T07:33:41.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.804+0000 7fa4a4664740 1 client.15075 shutdown 2024-09-19T07:33:41.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.804+0000 7fa4a4664740 1 -- 172.21.15.159:0/4147820268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 msgr2=0x562d917cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:41.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.804+0000 7fa4a4664740 1 --2- 172.21.15.159:0/4147820268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 0x562d917cb4f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009900 tx=0x7fa48c004440 comp rx=0 tx=0).stop 2024-09-19T07:33:41.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.804+0000 7fa4a4664740 1 -- 172.21.15.159:0/4147820268 shutdown_connections 2024-09-19T07:33:41.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.804+0000 7fa4a4664740 1 --2- 172.21.15.159:0/4147820268 >> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] conn(0x562d917df3e0 0x562d917e0850 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.804+0000 7fa4a4664740 1 --2- 172.21.15.159:0/4147820268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917ce510 0x562d917cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.804+0000 7fa4a4664740 1 --2- 172.21.15.159:0/4147820268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 0x562d917cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.804+0000 7fa4a4664740 1 --2- 172.21.15.159:0/4147820268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 0x562d917cb4f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.804+0000 7fa4a4664740 1 -- 172.21.15.159:0/4147820268 >> 172.21.15.159:0/4147820268 conn(0x562d917bf8c0 msgr2=0x562d917c55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:41.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.804+0000 7fa488c00640 0 client.15075 ms_handle_reset on v2:172.21.15.203:6838/3862781462 2024-09-19T07:33:41.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.804+0000 7fa4a4664740 1 -- 172.21.15.159:0/4147820268 shutdown_connections 2024-09-19T07:33:41.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.804+0000 7fa4a4664740 1 -- 172.21.15.159:0/4147820268 wait complete. 2024-09-19T07:33:41.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.804+0000 7fa4a4664740 20 client.15075 trim_cache size 0 max 16384 2024-09-19T07:33:41.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.822+0000 7fa4a4664740 1 Processor -- start 2024-09-19T07:33:41.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.823+0000 7fa4a4664740 1 -- start start 2024-09-19T07:33:41.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.823+0000 7fa4a4664740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917c3330 0x562d917df3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.823+0000 7fa4a4664740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3c50 0x562d917e8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.823+0000 7fa4a4664740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917ce510 0x562d917e9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.823+0000 7fa4a4664740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562d917deb90 con 0x562d917c3c50 2024-09-19T07:33:41.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.823+0000 7fa4a4664740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562d917df260 con 0x562d917ce510 2024-09-19T07:33:41.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.823+0000 7fa4a4664740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562d917df930 con 0x562d917c3330 2024-09-19T07:33:41.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.823+0000 7fa492000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3c50 0x562d917e8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.823+0000 7fa492000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3c50 0x562d917e8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37452/0 (socket says 172.21.15.159:37452) 2024-09-19T07:33:41.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.823+0000 7fa492000640 1 -- 172.21.15.159:0/925038375 learned_addr learned my addr 172.21.15.159:0/925038375 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:41.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.823+0000 7fa49b400640 1 --2- 172.21.15.159:0/925038375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917ce510 0x562d917e9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.823+0000 7fa49aa00640 1 --2- 172.21.15.159:0/925038375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917c3330 0x562d917df3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.824+0000 7fa492000640 1 -- 172.21.15.159:0/925038375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917c3330 msgr2=0x562d917df3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:41.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.824+0000 7fa492000640 1 --2- 172.21.15.159:0/925038375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917c3330 0x562d917df3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.824+0000 7fa492000640 1 -- 172.21.15.159:0/925038375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917ce510 msgr2=0x562d917e9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:41.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.824+0000 7fa492000640 1 --2- 172.21.15.159:0/925038375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917ce510 0x562d917e9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.824+0000 7fa492000640 1 -- 172.21.15.159:0/925038375 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d917cc870 con 0x562d917c3c50 2024-09-19T07:33:41.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.824+0000 7fa492000640 1 --2- 172.21.15.159:0/925038375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3c50 0x562d917e8b30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa484000c00 tx=0x7fa4840039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4069813e9ba3a03c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:41.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.824+0000 7fa48b400640 1 -- 172.21.15.159:0/925038375 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4840be9b0 con 0x562d917c3c50 2024-09-19T07:33:41.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.824+0000 7fa48b400640 1 -- 172.21.15.159:0/925038375 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4840d3340 con 0x562d917c3c50 2024-09-19T07:33:41.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.824+0000 7fa48b400640 1 -- 172.21.15.159:0/925038375 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4840bbdb0 con 0x562d917c3c50 2024-09-19T07:33:41.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.825+0000 7fa4a4664740 1 -- 172.21.15.159:0/925038375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3c50 msgr2=0x562d917e8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:41.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.825+0000 7fa4a4664740 1 --2- 172.21.15.159:0/925038375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3c50 0x562d917e8b30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa484000c00 tx=0x7fa4840039f0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.825+0000 7fa4a4664740 1 -- 172.21.15.159:0/925038375 shutdown_connections 2024-09-19T07:33:41.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.825+0000 7fa4a4664740 1 --2- 172.21.15.159:0/925038375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917ce510 0x562d917e9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.825+0000 7fa4a4664740 1 --2- 172.21.15.159:0/925038375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3c50 0x562d917e8b30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.825+0000 7fa4a4664740 1 --2- 172.21.15.159:0/925038375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917c3330 0x562d917df3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.825+0000 7fa4a4664740 1 -- 172.21.15.159:0/925038375 >> 172.21.15.159:0/925038375 conn(0x562d917bf8c0 msgr2=0x562d917bfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:41.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.825+0000 7fa4a4664740 1 -- 172.21.15.159:0/925038375 shutdown_connections 2024-09-19T07:33:41.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.825+0000 7fa4a4664740 1 -- 172.21.15.159:0/925038375 wait complete. 2024-09-19T07:33:41.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.825+0000 7fa4a4664740 1 -- start start 2024-09-19T07:33:41.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.826+0000 7fa4a4664740 1 Processor -- start 2024-09-19T07:33:41.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.826+0000 7fa4a4664740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 0x562d917e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.826+0000 7fa4a4664740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 0x562d917e4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.826+0000 7fa4a4664740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917ce510 0x562d918502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.826+0000 7fa4a4664740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562d917e06d0 con 0x562d917c3330 2024-09-19T07:33:41.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.826+0000 7fa4a4664740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562d917e0000 con 0x562d917c3c50 2024-09-19T07:33:41.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.826+0000 7fa4a4664740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562d917e0da0 con 0x562d917ce510 2024-09-19T07:33:41.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.826+0000 7fa4a4664740 10 client.0 _ll_register_callbacks cb 0x7fa48c0040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:33:41.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.826+0000 7fa49aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 0x562d917e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.826+0000 7fa492000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 0x562d917e4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.826+0000 7fa49aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 0x562d917e3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37462/0 (socket says 172.21.15.159:37462) 2024-09-19T07:33:41.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.826+0000 7fa49aa00640 1 -- 172.21.15.159:0/3975020241 learned_addr learned my addr 172.21.15.159:0/3975020241 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:41.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.826+0000 7fa49b400640 1 --2- 172.21.15.159:0/3975020241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917ce510 0x562d918502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.827+0000 7fa492000640 1 -- 172.21.15.159:0/3975020241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917ce510 msgr2=0x562d918502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:41.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.827+0000 7fa492000640 1 --2- 172.21.15.159:0/3975020241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917ce510 0x562d918502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.827+0000 7fa492000640 1 -- 172.21.15.159:0/3975020241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 msgr2=0x562d917e3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:41.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.827+0000 7fa492000640 1 --2- 172.21.15.159:0/3975020241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 0x562d917e3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.827+0000 7fa492000640 1 -- 172.21.15.159:0/3975020241 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d91850830 con 0x562d917c3c50 2024-09-19T07:33:41.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.827+0000 7fa492000640 1 --2- 172.21.15.159:0/3975020241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 0x562d917e4450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x562d917be0c0 tx=0x7fa48400df10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e1b6eb01d946c2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:41.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.827+0000 7fa488c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-19T07:33:41.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.827+0000 7fa488c00640 1 -- 172.21.15.159:0/3975020241 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484014030 con 0x562d917c3c50 2024-09-19T07:33:41.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.827+0000 7fa4a4664740 1 -- 172.21.15.159:0/3975020241 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4840d3340 con 0x562d917c3c50 2024-09-19T07:33:41.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.827+0000 7fa488c00640 1 -- 172.21.15.159:0/3975020241 <== mon.2 v2:172.21.15.189:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa4840035f0 con 0x562d917c3c50 2024-09-19T07:33:41.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.828+0000 7fa488c00640 1 -- 172.21.15.159:0/3975020241 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa484020070 con 0x562d917c3c50 2024-09-19T07:33:41.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.828+0000 7fa4a4664740 10 client.25112 fetch_fsmap learned FSMap version 25 2024-09-19T07:33:41.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.828+0000 7fa4a4664740 1 -- 172.21.15.159:0/3975020241 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d91851a90 con 0x562d917c3c50 2024-09-19T07:33:41.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.828+0000 7fa488c00640 1 -- 172.21.15.159:0/3975020241 <== mon.2 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4840d3340 con 0x562d917c3c50 2024-09-19T07:33:41.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.828+0000 7fa488c00640 1 -- 172.21.15.159:0/3975020241 <== mon.2 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa4840cd2a0 con 0x562d917c3c50 2024-09-19T07:33:41.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.828+0000 7fa4a4664740 10 client.25112 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T07:33:41.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.828+0000 7fa4a4664740 10 client.25112 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T07:33:41.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.828+0000 7fa4a4664740 20 client.25112 populate_metadata read hostname 'smithi159' 2024-09-19T07:33:41.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.828+0000 7fa4a4664740 1 --2- 172.21.15.159:0/3975020241 >> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] conn(0x562d91855ef0 0x562d91854c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:41.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.828+0000 7fa4a4664740 4 client.25112 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:33:41.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.828+0000 7fa4a4664740 1 -- 172.21.15.159:0/3975020241 --> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] -- command(tid 0: {"prefix": "status"}) -- 0x562d91730b30 con 0x562d91855ef0 2024-09-19T07:33:41.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.829+0000 7fa49aa00640 1 --2- 172.21.15.159:0/3975020241 >> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] conn(0x562d91855ef0 0x562d91854c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:41.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.829+0000 7fa49aa00640 1 --2- 172.21.15.159:0/3975020241 >> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] conn(0x562d91855ef0 0x562d91854c00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e7b4cee880288884 server_cookie=f1c08a986897e3b2 in_seq=0 out_seq=0 2024-09-19T07:33:41.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.829+0000 7fa488c00640 10 client.25112 ms_handle_connect on v2:172.21.15.203:6838/3862781462 2024-09-19T07:33:41.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.829+0000 7fa488c00640 1 -- 172.21.15.159:0/3975020241 <== mds.1 v2:172.21.15.203:6838/3862781462 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x562d91730b30 con 0x562d91855ef0 2024-09-19T07:33:41.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.829+0000 7fa488c00640 10 client.25112 handle_command_reply: tid=0 2024-09-19T07:33:41.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.829+0000 7fa4a4664740 1 client.25112 shutdown 2024-09-19T07:33:41.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.829+0000 7fa4a4664740 1 -- 172.21.15.159:0/3975020241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 msgr2=0x562d917e4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:41.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.829+0000 7fa4a4664740 1 --2- 172.21.15.159:0/3975020241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 0x562d917e4450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x562d917be0c0 tx=0x7fa48400df10 comp rx=0 tx=0).stop 2024-09-19T07:33:41.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.830+0000 7fa4a4664740 1 -- 172.21.15.159:0/3975020241 shutdown_connections 2024-09-19T07:33:41.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.830+0000 7fa4a4664740 1 --2- 172.21.15.159:0/3975020241 >> [v2:172.21.15.203:6838/3862781462,v1:172.21.15.203:6839/3862781462] conn(0x562d91855ef0 0x562d91854c00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.830+0000 7fa4a4664740 1 --2- 172.21.15.159:0/3975020241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d917ce510 0x562d918502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.830+0000 7fa4a4664740 1 --2- 172.21.15.159:0/3975020241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d917c3c50 0x562d917e4450 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.830+0000 7fa4a4664740 1 --2- 172.21.15.159:0/3975020241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d917c3330 0x562d917e3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.830+0000 7fa488c00640 0 client.25112 ms_handle_reset on v2:172.21.15.203:6838/3862781462 2024-09-19T07:33:41.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.830+0000 7fa4a4664740 1 -- 172.21.15.159:0/3975020241 >> 172.21.15.159:0/3975020241 conn(0x562d917bf8c0 msgr2=0x562d917c4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:41.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.830+0000 7fa4a4664740 1 -- 172.21.15.159:0/3975020241 shutdown_connections 2024-09-19T07:33:41.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.830+0000 7fa4a4664740 1 -- 172.21.15.159:0/3975020241 wait complete. 2024-09-19T07:33:41.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.830+0000 7fa4a4664740 20 client.25112 trim_cache size 0 max 16384 2024-09-19T07:33:41.835 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-19T07:33:41.835 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "c9d18048-7657-11ef-bceb-c7b262605968", 2024-09-19T07:33:41.835 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 1, 2024-09-19T07:33:41.835 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 24514, 2024-09-19T07:33:41.835 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-09-19T07:33:41.835 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-09-19T07:33:41.836 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-09-19T07:33:41.836 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 111.954302201, 2024-09-19T07:33:41.836 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 25, 2024-09-19T07:33:41.836 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 100, 2024-09-19T07:33:41.836 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 98, 2024-09-19T07:33:41.836 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 183.30724753800001 2024-09-19T07:33:41.836 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-19T07:33:41.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.830+0000 7fa4a1c00640 1 -- 172.21.15.159:0/3811336671 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa4800a32c0 msgr2=0x7fa4800a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:41.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.830+0000 7fa4a1c00640 1 --2- 172.21.15.159:0/3811336671 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa4800a32c0 0x7fa4800a5780 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa4940116f0 tx=0x7fa494009500 comp rx=0 tx=0).stop 2024-09-19T07:33:41.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.830+0000 7fa4a1c00640 1 -- 172.21.15.159:0/3811336671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa49c1049a0 msgr2=0x7fa49c19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:41.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.830+0000 7fa4a1c00640 1 --2- 172.21.15.159:0/3811336671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa49c1049a0 0x7fa49c19ab40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa484005fd0 tx=0x7fa48400dde0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.831+0000 7fa4a1c00640 1 -- 172.21.15.159:0/3811336671 shutdown_connections 2024-09-19T07:33:41.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.831+0000 7fa4a1c00640 1 --2- 172.21.15.159:0/3811336671 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa4800a32c0 0x7fa4800a5780 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.831+0000 7fa4a1c00640 1 --2- 172.21.15.159:0/3811336671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa49c109120 0x7fa49c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.831+0000 7fa4a1c00640 1 --2- 172.21.15.159:0/3811336671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa49c1049a0 0x7fa49c19ab40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.831+0000 7fa4a1c00640 1 --2- 172.21.15.159:0/3811336671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa49c103ff0 0x7fa49c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:41.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.831+0000 7fa4a1c00640 1 -- 172.21.15.159:0/3811336671 >> 172.21.15.159:0/3811336671 conn(0x7fa49c0ffa90 msgr2=0x7fa49c100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:41.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.831+0000 7fa4a1c00640 1 -- 172.21.15.159:0/3811336671 shutdown_connections 2024-09-19T07:33:41.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:41.831+0000 7fa4a1c00640 1 -- 172.21.15.159:0/3811336671 wait complete. 2024-09-19T07:33:42.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:41 smithi159 ceph-mon[30472]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T07:33:42.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:41 smithi203 ceph-mon[35335]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T07:33:42.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:41 smithi189 ceph-mon[34851]: pgmap v395: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T07:33:42.795 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T07:33:43.166 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:43.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.723+0000 7f1965a00640 1 -- 172.21.15.159:0/2727823645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19601049c0 msgr2=0x7f1960108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:43.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.723+0000 7f1965a00640 1 --2- 172.21.15.159:0/2727823645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19601049c0 0x7f1960108a10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f194c009860 tx=0x7f194c02a340 comp rx=0 tx=0).stop 2024-09-19T07:33:43.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.723+0000 7f1965a00640 1 -- 172.21.15.159:0/2727823645 shutdown_connections 2024-09-19T07:33:43.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.723+0000 7f1965a00640 1 --2- 172.21.15.159:0/2727823645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1960109140 0x7f1960110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.723+0000 7f1965a00640 1 --2- 172.21.15.159:0/2727823645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19601049c0 0x7f1960108a10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.723+0000 7f1965a00640 1 --2- 172.21.15.159:0/2727823645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1960104010 0x7f19601043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.723+0000 7f1965a00640 1 -- 172.21.15.159:0/2727823645 >> 172.21.15.159:0/2727823645 conn(0x7f19600ffa50 msgr2=0x7f1960101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:43.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.723+0000 7f1965a00640 1 -- 172.21.15.159:0/2727823645 shutdown_connections 2024-09-19T07:33:43.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.723+0000 7f1965a00640 1 -- 172.21.15.159:0/2727823645 wait complete. 2024-09-19T07:33:43.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.724+0000 7f1965a00640 1 Processor -- start 2024-09-19T07:33:43.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.724+0000 7f1965a00640 1 -- start start 2024-09-19T07:33:43.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.725+0000 7f1965a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1960104010 0x7f196019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.725+0000 7f1965a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19601049c0 0x7f196019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.725+0000 7f1965a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1960109140 0x7f196019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.725+0000 7f1965a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1960114140 con 0x7f19601049c0 2024-09-19T07:33:43.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.725+0000 7f1965a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1960113fc0 con 0x7f1960104010 2024-09-19T07:33:43.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.725+0000 7f1965a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f19601142c0 con 0x7f1960109140 2024-09-19T07:33:43.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.725+0000 7f195ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1960104010 0x7f196019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.725+0000 7f195e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19601049c0 0x7f196019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.725+0000 7f195ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1960104010 0x7f196019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:34096/0 (socket says 172.21.15.159:34096) 2024-09-19T07:33:43.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.725+0000 7f195ea00640 1 -- 172.21.15.159:0/1798249361 learned_addr learned my addr 172.21.15.159:0/1798249361 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:43.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.725+0000 7f195f400640 1 --2- 172.21.15.159:0/1798249361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1960109140 0x7f196019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.726+0000 7f195ea00640 1 -- 172.21.15.159:0/1798249361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1960109140 msgr2=0x7f196019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:43.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.726+0000 7f195ea00640 1 --2- 172.21.15.159:0/1798249361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1960109140 0x7f196019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.726+0000 7f195ea00640 1 -- 172.21.15.159:0/1798249361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19601049c0 msgr2=0x7f196019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:43.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.726+0000 7f195ea00640 1 --2- 172.21.15.159:0/1798249361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19601049c0 0x7f196019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.726+0000 7f195ea00640 1 -- 172.21.15.159:0/1798249361 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196019f610 con 0x7f1960104010 2024-09-19T07:33:43.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.726+0000 7f195f400640 1 --2- 172.21.15.159:0/1798249361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1960109140 0x7f196019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:43.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.726+0000 7f195ea00640 1 --2- 172.21.15.159:0/1798249361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1960104010 0x7f196019a660 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f195000e9c0 tx=0x7f195000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:43.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.726+0000 7f1957400640 1 -- 172.21.15.159:0/1798249361 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195000ccc0 con 0x7f1960104010 2024-09-19T07:33:43.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.726+0000 7f1965a00640 1 -- 172.21.15.159:0/1798249361 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196019f900 con 0x7f1960104010 2024-09-19T07:33:43.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.726+0000 7f1957400640 1 -- 172.21.15.159:0/1798249361 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1950005500 con 0x7f1960104010 2024-09-19T07:33:43.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.726+0000 7f1957400640 1 -- 172.21.15.159:0/1798249361 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950010430 con 0x7f1960104010 2024-09-19T07:33:43.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.727+0000 7f1965a00640 1 -- 172.21.15.159:0/1798249361 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19601a71e0 con 0x7f1960104010 2024-09-19T07:33:43.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.730+0000 7f1957400640 1 -- 172.21.15.159:0/1798249361 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f19500046c0 con 0x7f1960104010 2024-09-19T07:33:43.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.730+0000 7f19685c6740 1 Processor -- start 2024-09-19T07:33:43.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.730+0000 7f19685c6740 1 -- start start 2024-09-19T07:33:43.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.731+0000 7f19685c6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c01330 0x5639a9c01710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.731+0000 7f19685c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01c50 0x5639a9c0bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.731+0000 7f19685c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c0c510 0x5639a9c0e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.731+0000 7f19685c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5639a9b670b0 con 0x5639a9c0c510 2024-09-19T07:33:43.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.731+0000 7f19685c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5639a9b0eab0 con 0x5639a9c01c50 2024-09-19T07:33:43.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.731+0000 7f19685c6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5639a9b63070 con 0x5639a9c01330 2024-09-19T07:33:43.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.731+0000 7f195f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c01330 0x5639a9c01710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.731+0000 7f195f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c01330 0x5639a9c01710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:38744/0 (socket says 172.21.15.159:38744) 2024-09-19T07:33:43.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.731+0000 7f195f400640 1 -- 172.21.15.159:0/3116235172 learned_addr learned my addr 172.21.15.159:0/3116235172 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:43.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.731+0000 7f195ea00640 1 --2- 172.21.15.159:0/3116235172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01c50 0x5639a9c0bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.731+0000 7f195e000640 1 --2- 172.21.15.159:0/3116235172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c0c510 0x5639a9c0e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.731+0000 7f195f400640 1 -- 172.21.15.159:0/3116235172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01c50 msgr2=0x5639a9c0bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:43.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.731+0000 7f195f400640 1 --2- 172.21.15.159:0/3116235172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01c50 0x5639a9c0bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.732+0000 7f195f400640 1 -- 172.21.15.159:0/3116235172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c0c510 msgr2=0x5639a9c0e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:43.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.732+0000 7f195f400640 1 --2- 172.21.15.159:0/3116235172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c0c510 0x5639a9c0e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.732+0000 7f195f400640 1 -- 172.21.15.159:0/3116235172 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a9b1b4a0 con 0x5639a9c01330 2024-09-19T07:33:43.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.732+0000 7f195f400640 1 --2- 172.21.15.159:0/3116235172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c01330 0x5639a9c01710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1958011790 tx=0x7f1958011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5952de2639a34cda server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:43.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.732+0000 7f1957400640 1 --2- 172.21.15.159:0/1798249361 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f19440a3240 0x7f19440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.732+0000 7f194be00640 1 -- 172.21.15.159:0/3116235172 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958003c30 con 0x5639a9c01330 2024-09-19T07:33:43.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.732+0000 7f194be00640 1 -- 172.21.15.159:0/3116235172 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f195800fe80 con 0x5639a9c01330 2024-09-19T07:33:43.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.732+0000 7f194be00640 1 -- 172.21.15.159:0/3116235172 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19580193e0 con 0x5639a9c01330 2024-09-19T07:33:43.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.732+0000 7f1957400640 1 -- 172.21.15.159:0/1798249361 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f19500cff70 con 0x7f1960104010 2024-09-19T07:33:43.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.732+0000 7f195ea00640 1 --2- 172.21.15.159:0/1798249361 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f19440a3240 0x7f19440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.733+0000 7f19685c6740 1 -- 172.21.15.159:0/3116235172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c01330 msgr2=0x5639a9c01710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:43.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.733+0000 7f19685c6740 1 --2- 172.21.15.159:0/3116235172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c01330 0x5639a9c01710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1958011790 tx=0x7f1958011c50 comp rx=0 tx=0).stop 2024-09-19T07:33:43.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.733+0000 7f19685c6740 1 -- 172.21.15.159:0/3116235172 shutdown_connections 2024-09-19T07:33:43.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.733+0000 7f19685c6740 1 --2- 172.21.15.159:0/3116235172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c0c510 0x5639a9c0e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.733+0000 7f19685c6740 1 --2- 172.21.15.159:0/3116235172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01c50 0x5639a9c0bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.733+0000 7f19685c6740 1 --2- 172.21.15.159:0/3116235172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c01330 0x5639a9c01710 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.733+0000 7f19685c6740 1 -- 172.21.15.159:0/3116235172 >> 172.21.15.159:0/3116235172 conn(0x5639a9bfd8c0 msgr2=0x5639a9bfdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:43.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.733+0000 7f19685c6740 1 -- 172.21.15.159:0/3116235172 shutdown_connections 2024-09-19T07:33:43.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.733+0000 7f19685c6740 1 -- 172.21.15.159:0/3116235172 wait complete. 2024-09-19T07:33:43.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.733+0000 7f195ea00640 1 --2- 172.21.15.159:0/1798249361 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f19440a3240 0x7f19440a5700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f19500a48c0 tx=0x7f1950009f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:43.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.734+0000 7f19685c6740 1 -- start start 2024-09-19T07:33:43.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.734+0000 7f19685c6740 1 Processor -- start 2024-09-19T07:33:43.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.734+0000 7f19685c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01330 0x5639a9c09540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.734+0000 7f19685c6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c01c50 0x5639a9c09a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.734+0000 7f19685c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c0c510 0x5639a9c09fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.734+0000 7f19685c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5639a9c0f060 con 0x5639a9c0c510 2024-09-19T07:33:43.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.734+0000 7f19685c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5639a9aef920 con 0x5639a9c01330 2024-09-19T07:33:43.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.734+0000 7f19685c6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5639a9c0f1e0 con 0x5639a9c01c50 2024-09-19T07:33:43.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.734+0000 7f19685c6740 10 client.0 _ll_register_callbacks cb 0x5639a9b138a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:33:43.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.735+0000 7f195e000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c01c50 0x5639a9c09a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.735+0000 7f195f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01330 0x5639a9c09540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.735+0000 7f195e000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c01c50 0x5639a9c09a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:38754/0 (socket says 172.21.15.159:38754) 2024-09-19T07:33:43.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.735+0000 7f195e000640 1 -- 172.21.15.159:0/4096837926 learned_addr learned my addr 172.21.15.159:0/4096837926 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:43.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.735+0000 7f195f400640 1 --2- 172.21.15.159:0/4096837926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c0c510 0x5639a9c09fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.735+0000 7f195e000640 1 -- 172.21.15.159:0/4096837926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01330 msgr2=0x5639a9c09540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:43.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.735+0000 7f195e000640 1 --2- 172.21.15.159:0/4096837926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01330 0x5639a9c09540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.735+0000 7f195e000640 1 -- 172.21.15.159:0/4096837926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c0c510 msgr2=0x5639a9c09fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:43.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.735+0000 7f195e000640 1 --2- 172.21.15.159:0/4096837926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c0c510 0x5639a9c09fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.735+0000 7f195e000640 1 -- 172.21.15.159:0/4096837926 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a98d82b0 con 0x5639a9c01c50 2024-09-19T07:33:43.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.735+0000 7f195e000640 1 --2- 172.21.15.159:0/4096837926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c01c50 0x5639a9c09a80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f194c005e00 tx=0x7f194c0095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c9f0db52a00432a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:43.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.735+0000 7f1948c00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-09-19T07:33:43.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.736+0000 7f1948c00640 1 -- 172.21.15.159:0/4096837926 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c037040 con 0x5639a9c01c50 2024-09-19T07:33:43.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.736+0000 7f19685c6740 1 -- 172.21.15.159:0/4096837926 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5639a9b74620 con 0x5639a9c01c50 2024-09-19T07:33:43.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.736+0000 7f1948c00640 1 -- 172.21.15.159:0/4096837926 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f194c030550 con 0x5639a9c01c50 2024-09-19T07:33:43.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.736+0000 7f1948c00640 1 -- 172.21.15.159:0/4096837926 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f194c04b050 con 0x5639a9c01c50 2024-09-19T07:33:43.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.736+0000 7f19685c6740 10 client.25141 fetch_fsmap learned FSMap version 25 2024-09-19T07:33:43.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.736+0000 7f19685c6740 1 -- 172.21.15.159:0/4096837926 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5639a9c1ae20 con 0x5639a9c01c50 2024-09-19T07:33:43.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.736+0000 7f1948c00640 1 -- 172.21.15.159:0/4096837926 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5639a9b74620 con 0x5639a9c01c50 2024-09-19T07:33:43.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.736+0000 7f1948c00640 1 -- 172.21.15.159:0/4096837926 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f194c032440 con 0x5639a9c01c50 2024-09-19T07:33:43.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.736+0000 7f19685c6740 10 client.25141 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T07:33:43.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.737+0000 7f19685c6740 10 client.25141 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T07:33:43.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.737+0000 7f19685c6740 20 client.25141 populate_metadata read hostname 'smithi159' 2024-09-19T07:33:43.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.737+0000 7f19685c6740 1 --2- 172.21.15.159:0/4096837926 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x5639a9c1d420 0x5639a9c1e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.737+0000 7f19685c6740 4 client.25141 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:33:43.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.737+0000 7f19685c6740 1 -- 172.21.15.159:0/4096837926 --> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639a9b6eb30 con 0x5639a9c1d420 2024-09-19T07:33:43.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.737+0000 7f195f400640 1 --2- 172.21.15.159:0/4096837926 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x5639a9c1d420 0x5639a9c1e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.737+0000 7f195f400640 1 --2- 172.21.15.159:0/4096837926 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x5639a9c1d420 0x5639a9c1e890 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f9875a58648112d server_cookie=10b866ad31d887cf in_seq=0 out_seq=0 2024-09-19T07:33:43.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.737+0000 7f1948c00640 10 client.25141 ms_handle_connect on v2:172.21.15.189:6838/2361818663 2024-09-19T07:33:43.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.738+0000 7f1948c00640 1 -- 172.21.15.159:0/4096837926 <== mds.0 v2:172.21.15.189:6838/2361818663 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5639a9b6eb30 con 0x5639a9c1d420 2024-09-19T07:33:43.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.738+0000 7f1948c00640 10 client.25141 handle_command_reply: tid=0 2024-09-19T07:33:43.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.738+0000 7f19685c6740 1 client.25141 shutdown 2024-09-19T07:33:43.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.738+0000 7f19685c6740 1 -- 172.21.15.159:0/4096837926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c01c50 msgr2=0x5639a9c09a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:43.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.738+0000 7f19685c6740 1 --2- 172.21.15.159:0/4096837926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c01c50 0x5639a9c09a80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f194c005e00 tx=0x7f194c0095b0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.738+0000 7f19685c6740 1 -- 172.21.15.159:0/4096837926 shutdown_connections 2024-09-19T07:33:43.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.738+0000 7f19685c6740 1 --2- 172.21.15.159:0/4096837926 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x5639a9c1d420 0x5639a9c1e890 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.738+0000 7f19685c6740 1 --2- 172.21.15.159:0/4096837926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c0c510 0x5639a9c09fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.738+0000 7f19685c6740 1 --2- 172.21.15.159:0/4096837926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c01c50 0x5639a9c09a80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.738+0000 7f19685c6740 1 --2- 172.21.15.159:0/4096837926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01330 0x5639a9c09540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.738+0000 7f1948c00640 0 client.25141 ms_handle_reset on v2:172.21.15.189:6838/2361818663 2024-09-19T07:33:43.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.738+0000 7f19685c6740 1 -- 172.21.15.159:0/4096837926 >> 172.21.15.159:0/4096837926 conn(0x5639a9bfd8c0 msgr2=0x5639a9c035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:43.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.739+0000 7f19685c6740 1 -- 172.21.15.159:0/4096837926 shutdown_connections 2024-09-19T07:33:43.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.739+0000 7f19685c6740 1 -- 172.21.15.159:0/4096837926 wait complete. 2024-09-19T07:33:43.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.739+0000 7f19685c6740 20 client.25141 trim_cache size 0 max 16384 2024-09-19T07:33:43.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.757+0000 7f19685c6740 1 Processor -- start 2024-09-19T07:33:43.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.757+0000 7f19685c6740 1 -- start start 2024-09-19T07:33:43.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.758+0000 7f19685c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c01330 0x5639a9c09e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.758+0000 7f19685c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01c50 0x5639a9c06820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.758+0000 7f19685c6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c0c510 0x5639a9c06d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.758+0000 7f19685c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5639a9c1cbd0 con 0x5639a9c01330 2024-09-19T07:33:43.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.758+0000 7f19685c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5639a9c1d2a0 con 0x5639a9c01c50 2024-09-19T07:33:43.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.758+0000 7f19685c6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5639a9c1d970 con 0x5639a9c0c510 2024-09-19T07:33:43.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.758+0000 7f195f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c01330 0x5639a9c09e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.758+0000 7f195f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c01330 0x5639a9c09e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47716/0 (socket says 172.21.15.159:47716) 2024-09-19T07:33:43.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.758+0000 7f195f400640 1 -- 172.21.15.159:0/2171576518 learned_addr learned my addr 172.21.15.159:0/2171576518 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:43.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.758+0000 7f195e000640 1 --2- 172.21.15.159:0/2171576518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01c50 0x5639a9c06820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.758+0000 7f195f400640 1 --2- 172.21.15.159:0/2171576518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c0c510 0x5639a9c06d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.758+0000 7f195f400640 1 -- 172.21.15.159:0/2171576518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c0c510 msgr2=0x5639a9c06d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:43.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.758+0000 7f195f400640 1 --2- 172.21.15.159:0/2171576518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c0c510 0x5639a9c06d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.758+0000 7f195f400640 1 -- 172.21.15.159:0/2171576518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01c50 msgr2=0x5639a9c06820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:43.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.758+0000 7f195f400640 1 --2- 172.21.15.159:0/2171576518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01c50 0x5639a9c06820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.758+0000 7f195f400640 1 -- 172.21.15.159:0/2171576518 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a9c1e4f0 con 0x5639a9c01330 2024-09-19T07:33:43.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.758+0000 7f195e000640 1 --2- 172.21.15.159:0/2171576518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01c50 0x5639a9c06820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:43.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.759+0000 7f195f400640 1 --2- 172.21.15.159:0/2171576518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c01330 0x5639a9c09e30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1958004590 tx=0x7f19580045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d100fb2deade4ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.759+0000 7f194b400640 1 -- 172.21.15.159:0/2171576518 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195802f020 con 0x5639a9c01330 2024-09-19T07:33:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.759+0000 7f194b400640 1 -- 172.21.15.159:0/2171576518 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1958003a30 con 0x5639a9c01330 2024-09-19T07:33:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.759+0000 7f194b400640 1 -- 172.21.15.159:0/2171576518 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958005030 con 0x5639a9c01330 2024-09-19T07:33:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.759+0000 7f19685c6740 1 -- 172.21.15.159:0/2171576518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c01330 msgr2=0x5639a9c09e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.759+0000 7f19685c6740 1 --2- 172.21.15.159:0/2171576518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c01330 0x5639a9c09e30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1958004590 tx=0x7f19580045c0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.760+0000 7f19685c6740 1 -- 172.21.15.159:0/2171576518 shutdown_connections 2024-09-19T07:33:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.760+0000 7f19685c6740 1 --2- 172.21.15.159:0/2171576518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c0c510 0x5639a9c06d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.760+0000 7f19685c6740 1 --2- 172.21.15.159:0/2171576518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01c50 0x5639a9c06820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.760+0000 7f19685c6740 1 --2- 172.21.15.159:0/2171576518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c01330 0x5639a9c09e30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.760+0000 7f19685c6740 1 -- 172.21.15.159:0/2171576518 >> 172.21.15.159:0/2171576518 conn(0x5639a9bfd8c0 msgr2=0x5639a9c07e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.760+0000 7f19685c6740 1 -- 172.21.15.159:0/2171576518 shutdown_connections 2024-09-19T07:33:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.760+0000 7f19685c6740 1 -- 172.21.15.159:0/2171576518 wait complete. 2024-09-19T07:33:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.760+0000 7f19685c6740 1 -- start start 2024-09-19T07:33:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.760+0000 7f19685c6740 1 Processor -- start 2024-09-19T07:33:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.761+0000 7f19685c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c01330 0x5639a9c26af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.761+0000 7f19685c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01c50 0x5639a9c27030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.761+0000 7f19685c6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c0c510 0x5639a9c21aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.761+0000 7f19685c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5639a9c1e710 con 0x5639a9c01330 2024-09-19T07:33:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.761+0000 7f19685c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5639a9c1e040 con 0x5639a9c01c50 2024-09-19T07:33:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.761+0000 7f19685c6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5639a9c1ede0 con 0x5639a9c0c510 2024-09-19T07:33:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.761+0000 7f19685c6740 10 client.0 _ll_register_callbacks cb 0x7f194c0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:33:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.761+0000 7f195f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c01330 0x5639a9c26af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.761+0000 7f195f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c01330 0x5639a9c26af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47724/0 (socket says 172.21.15.159:47724) 2024-09-19T07:33:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.761+0000 7f195f400640 1 -- 172.21.15.159:0/2240572187 learned_addr learned my addr 172.21.15.159:0/2240572187 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.761+0000 7f195e000640 1 --2- 172.21.15.159:0/2240572187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01c50 0x5639a9c27030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.761+0000 7f195f400640 1 --2- 172.21.15.159:0/2240572187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c0c510 0x5639a9c21aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.761+0000 7f195f400640 1 -- 172.21.15.159:0/2240572187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c0c510 msgr2=0x5639a9c21aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.761+0000 7f195f400640 1 --2- 172.21.15.159:0/2240572187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c0c510 0x5639a9c21aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.761+0000 7f195f400640 1 -- 172.21.15.159:0/2240572187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01c50 msgr2=0x5639a9c27030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:33:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.762+0000 7f195f400640 1 --2- 172.21.15.159:0/2240572187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01c50 0x5639a9c27030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.762+0000 7f195f400640 1 -- 172.21.15.159:0/2240572187 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a9c276f0 con 0x5639a9c01330 2024-09-19T07:33:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.762+0000 7f195e000640 1 --2- 172.21.15.159:0/2240572187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01c50 0x5639a9c27030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.762+0000 7f195f400640 1 --2- 172.21.15.159:0/2240572187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c01330 0x5639a9c26af0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1958004c60 tx=0x7f1958004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8255277943aa17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.762+0000 7f1948c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-19T07:33:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.762+0000 7f1948c00640 1 -- 172.21.15.159:0/2240572187 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195802f030 con 0x5639a9c01330 2024-09-19T07:33:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.762+0000 7f1948c00640 1 -- 172.21.15.159:0/2240572187 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f1958003da0 con 0x5639a9c01330 2024-09-19T07:33:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.762+0000 7f19685c6740 1 -- 172.21.15.159:0/2240572187 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1958003a30 con 0x5639a9c01330 2024-09-19T07:33:43.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.762+0000 7f1948c00640 1 -- 172.21.15.159:0/2240572187 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f195800f940 con 0x5639a9c01330 2024-09-19T07:33:43.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.762+0000 7f19685c6740 10 client.15099 fetch_fsmap learned FSMap version 25 2024-09-19T07:33:43.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.762+0000 7f19685c6740 1 -- 172.21.15.159:0/2240572187 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5639a9c880a0 con 0x5639a9c01330 2024-09-19T07:33:43.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.763+0000 7f1948c00640 1 -- 172.21.15.159:0/2240572187 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1958003a30 con 0x5639a9c01330 2024-09-19T07:33:43.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.763+0000 7f1948c00640 1 -- 172.21.15.159:0/2240572187 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1958018070 con 0x5639a9c01330 2024-09-19T07:33:43.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.763+0000 7f19685c6740 10 client.15099 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T07:33:43.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.763+0000 7f19685c6740 10 client.15099 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T07:33:43.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.763+0000 7f19685c6740 20 client.15099 populate_metadata read hostname 'smithi159' 2024-09-19T07:33:43.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.763+0000 7f19685c6740 1 --2- 172.21.15.159:0/2240572187 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x5639a9c8c420 0x5639a9c8b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:43.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.763+0000 7f19685c6740 4 client.15099 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:33:43.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.763+0000 7f19685c6740 1 -- 172.21.15.159:0/2240572187 --> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] -- command(tid 0: {"prefix": "status"}) -- 0x5639a9bfe470 con 0x5639a9c8c420 2024-09-19T07:33:43.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.763+0000 7f195e000640 1 --2- 172.21.15.159:0/2240572187 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x5639a9c8c420 0x5639a9c8b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:43.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.764+0000 7f195e000640 1 --2- 172.21.15.159:0/2240572187 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x5639a9c8c420 0x5639a9c8b130 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4824fb6a47593b3d server_cookie=55281fb4a5d3e59f in_seq=0 out_seq=0 2024-09-19T07:33:43.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.764+0000 7f1948c00640 10 client.15099 ms_handle_connect on v2:172.21.15.189:6838/2361818663 2024-09-19T07:33:43.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.764+0000 7f1948c00640 1 -- 172.21.15.159:0/2240572187 <== mds.0 v2:172.21.15.189:6838/2361818663 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5639a9bfe470 con 0x5639a9c8c420 2024-09-19T07:33:43.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.764+0000 7f1948c00640 10 client.15099 handle_command_reply: tid=0 2024-09-19T07:33:43.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.764+0000 7f19685c6740 1 client.15099 shutdown 2024-09-19T07:33:43.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.764+0000 7f19685c6740 1 -- 172.21.15.159:0/2240572187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c01330 msgr2=0x5639a9c26af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:43.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.764+0000 7f19685c6740 1 --2- 172.21.15.159:0/2240572187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c01330 0x5639a9c26af0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1958004c60 tx=0x7f1958004b60 comp rx=0 tx=0).stop 2024-09-19T07:33:43.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.764+0000 7f19685c6740 1 -- 172.21.15.159:0/2240572187 shutdown_connections 2024-09-19T07:33:43.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.764+0000 7f19685c6740 1 --2- 172.21.15.159:0/2240572187 >> [v2:172.21.15.189:6838/2361818663,v1:172.21.15.189:6839/2361818663] conn(0x5639a9c8c420 0x5639a9c8b130 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.764+0000 7f19685c6740 1 --2- 172.21.15.159:0/2240572187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639a9c0c510 0x5639a9c21aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.764+0000 7f19685c6740 1 --2- 172.21.15.159:0/2240572187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5639a9c01c50 0x5639a9c27030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.764+0000 7f19685c6740 1 --2- 172.21.15.159:0/2240572187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639a9c01330 0x5639a9c26af0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.764+0000 7f1948c00640 0 client.15099 ms_handle_reset on v2:172.21.15.189:6838/2361818663 2024-09-19T07:33:43.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.764+0000 7f19685c6740 1 -- 172.21.15.159:0/2240572187 >> 172.21.15.159:0/2240572187 conn(0x5639a9bfd8c0 msgr2=0x5639a9c02d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:43.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.765+0000 7f19685c6740 1 -- 172.21.15.159:0/2240572187 shutdown_connections 2024-09-19T07:33:43.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.765+0000 7f19685c6740 1 -- 172.21.15.159:0/2240572187 wait complete. 2024-09-19T07:33:43.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.765+0000 7f19685c6740 20 client.15099 trim_cache size 0 max 16384 2024-09-19T07:33:43.770 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-19T07:33:43.770 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "c9d18048-7657-11ef-bceb-c7b262605968", 2024-09-19T07:33:43.770 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 0, 2024-09-19T07:33:43.770 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 24542, 2024-09-19T07:33:43.770 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-09-19T07:33:43.770 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-09-19T07:33:43.770 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-09-19T07:33:43.770 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 159.33688550900001, 2024-09-19T07:33:43.770 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 25, 2024-09-19T07:33:43.770 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 100, 2024-09-19T07:33:43.770 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 98, 2024-09-19T07:33:43.770 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 174.131748249 2024-09-19T07:33:43.770 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-19T07:33:43.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.765+0000 7f1965a00640 1 -- 172.21.15.159:0/1798249361 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f19440a3240 msgr2=0x7f19440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:43.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.765+0000 7f1965a00640 1 --2- 172.21.15.159:0/1798249361 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f19440a3240 0x7f19440a5700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f19500a48c0 tx=0x7f1950009f90 comp rx=0 tx=0).stop 2024-09-19T07:33:43.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.765+0000 7f1965a00640 1 -- 172.21.15.159:0/1798249361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1960104010 msgr2=0x7f196019a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:43.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.765+0000 7f1965a00640 1 --2- 172.21.15.159:0/1798249361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1960104010 0x7f196019a660 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f195000e9c0 tx=0x7f195000ee80 comp rx=0 tx=0).stop 2024-09-19T07:33:43.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.766+0000 7f1965a00640 1 -- 172.21.15.159:0/1798249361 shutdown_connections 2024-09-19T07:33:43.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.766+0000 7f1965a00640 1 --2- 172.21.15.159:0/1798249361 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f19440a3240 0x7f19440a5700 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.766+0000 7f1965a00640 1 --2- 172.21.15.159:0/1798249361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1960109140 0x7f196019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.766+0000 7f1965a00640 1 --2- 172.21.15.159:0/1798249361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19601049c0 0x7f196019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.766+0000 7f1965a00640 1 --2- 172.21.15.159:0/1798249361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1960104010 0x7f196019a660 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:43.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.766+0000 7f1965a00640 1 -- 172.21.15.159:0/1798249361 >> 172.21.15.159:0/1798249361 conn(0x7f19600ffa50 msgr2=0x7f1960100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:43.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.766+0000 7f1965a00640 1 -- 172.21.15.159:0/1798249361 shutdown_connections 2024-09-19T07:33:43.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:43.766+0000 7f1965a00640 1 -- 172.21.15.159:0/1798249361 wait complete. 2024-09-19T07:33:44.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:43 smithi159 ceph-mon[30472]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T07:33:44.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:43 smithi203 ceph-mon[35335]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T07:33:44.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:43 smithi189 ceph-mon[34851]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-19T07:33:44.670 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T07:33:44.679 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T07:33:44.679 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T07:33:44.679 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T07:33:44.679 INFO:teuthology.run_tasks:Running task kclient... 2024-09-19T07:33:44.709 INFO:tasks.kclient:Mounting kernel clients... 2024-09-19T07:33:44.709 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-19T07:33:44.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:33:44.710 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-09-19T07:33:44.746 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-19T07:33:44.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:33:44.747 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T07:33:44.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:33:44.845 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link delete ceph-brx 2024-09-19T07:33:45.174 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-19T07:33:45.175 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T07:33:45.175 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:33:45.175 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T07:33:45.550 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:45.988 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:45 smithi159 ceph-mon[30472]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T07:33:46.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.102+0000 7fd425c00640 1 -- 172.21.15.159:0/717446837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd420106f10 msgr2=0x7fd42010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:46.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.102+0000 7fd425c00640 1 --2- 172.21.15.159:0/717446837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd420106f10 0x7fd42010ea90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd418009ca0 tx=0x7fd41802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:33:46.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.102+0000 7fd425c00640 1 -- 172.21.15.159:0/717446837 shutdown_connections 2024-09-19T07:33:46.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.102+0000 7fd425c00640 1 --2- 172.21.15.159:0/717446837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd420106f10 0x7fd42010ea90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:46.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.102+0000 7fd425c00640 1 --2- 172.21.15.159:0/717446837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd420102790 0x7fd4201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:46.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.102+0000 7fd425c00640 1 --2- 172.21.15.159:0/717446837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd420101de0 0x7fd4201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:46.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.102+0000 7fd425c00640 1 -- 172.21.15.159:0/717446837 >> 172.21.15.159:0/717446837 conn(0x7fd4200fd8c0 msgr2=0x7fd4200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:46.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.102+0000 7fd425c00640 1 -- 172.21.15.159:0/717446837 shutdown_connections 2024-09-19T07:33:46.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.102+0000 7fd425c00640 1 -- 172.21.15.159:0/717446837 wait complete. 2024-09-19T07:33:46.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.103+0000 7fd425c00640 1 Processor -- start 2024-09-19T07:33:46.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.103+0000 7fd425c00640 1 -- start start 2024-09-19T07:33:46.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.103+0000 7fd425c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd420101de0 0x7fd4200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:46.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.103+0000 7fd425c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd420102790 0x7fd420074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:46.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.104+0000 7fd425c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd420106f10 0x7fd420074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:46.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.104+0000 7fd425c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd420111fc0 con 0x7fd420101de0 2024-09-19T07:33:46.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.104+0000 7fd425c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd420111e40 con 0x7fd420102790 2024-09-19T07:33:46.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.104+0000 7fd425c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd420112140 con 0x7fd420106f10 2024-09-19T07:33:46.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.104+0000 7fd41ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd420101de0 0x7fd4200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:46.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.104+0000 7fd41ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd420101de0 0x7fd4200744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47752/0 (socket says 172.21.15.159:47752) 2024-09-19T07:33:46.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.104+0000 7fd41ea00640 1 -- 172.21.15.159:0/3033567157 learned_addr learned my addr 172.21.15.159:0/3033567157 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:46.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.104+0000 7fd41f400640 1 --2- 172.21.15.159:0/3033567157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd420106f10 0x7fd420074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:46.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.104+0000 7fd416000640 1 --2- 172.21.15.159:0/3033567157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd420102790 0x7fd420074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:46.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.104+0000 7fd41ea00640 1 -- 172.21.15.159:0/3033567157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd420106f10 msgr2=0x7fd420074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:46.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.104+0000 7fd41ea00640 1 --2- 172.21.15.159:0/3033567157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd420106f10 0x7fd420074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:46.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.104+0000 7fd41ea00640 1 -- 172.21.15.159:0/3033567157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd420102790 msgr2=0x7fd420074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:46.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.104+0000 7fd41ea00640 1 --2- 172.21.15.159:0/3033567157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd420102790 0x7fd420074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:46.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.104+0000 7fd41ea00640 1 -- 172.21.15.159:0/3033567157 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4201a6840 con 0x7fd420101de0 2024-09-19T07:33:46.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.105+0000 7fd41ea00640 1 --2- 172.21.15.159:0/3033567157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd420101de0 0x7fd4200744f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd41000e9f0 tx=0x7fd41000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:46.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.105+0000 7fd417e00640 1 -- 172.21.15.159:0/3033567157 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41000cd80 con 0x7fd420101de0 2024-09-19T07:33:46.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.105+0000 7fd417e00640 1 -- 172.21.15.159:0/3033567157 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd410010910 con 0x7fd420101de0 2024-09-19T07:33:46.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.105+0000 7fd417e00640 1 -- 172.21.15.159:0/3033567157 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410018990 con 0x7fd420101de0 2024-09-19T07:33:46.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.105+0000 7fd425c00640 1 -- 172.21.15.159:0/3033567157 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4201a6b30 con 0x7fd420101de0 2024-09-19T07:33:46.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.105+0000 7fd425c00640 1 -- 172.21.15.159:0/3033567157 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4201a7070 con 0x7fd420101de0 2024-09-19T07:33:46.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.106+0000 7fd425c00640 1 -- 172.21.15.159:0/3033567157 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4201021c0 con 0x7fd420101de0 2024-09-19T07:33:46.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.108+0000 7fd417e00640 1 -- 172.21.15.159:0/3033567157 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fd410010ac0 con 0x7fd420101de0 2024-09-19T07:33:46.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.109+0000 7fd417e00640 1 --2- 172.21.15.159:0/3033567157 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd4040a3240 0x7fd4040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:46.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.110+0000 7fd417e00640 1 -- 172.21.15.159:0/3033567157 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fd4100cfff0 con 0x7fd420101de0 2024-09-19T07:33:46.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.110+0000 7fd416000640 1 --2- 172.21.15.159:0/3033567157 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd4040a3240 0x7fd4040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:46.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.110+0000 7fd416000640 1 --2- 172.21.15.159:0/3033567157 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd4040a3240 0x7fd4040a5700 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd420075f80 tx=0x7fd408005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:46.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.114+0000 7fd417e00640 1 -- 172.21.15.159:0/3033567157 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd41009c2d0 con 0x7fd420101de0 2024-09-19T07:33:46.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:45 smithi203 ceph-mon[35335]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T07:33:46.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:45 smithi189 ceph-mon[34851]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-19T07:33:46.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.326+0000 7fd425c00640 1 -- 172.21.15.159:0/3033567157 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd42002d7c0 con 0x7fd420101de0 2024-09-19T07:33:46.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.327+0000 7fd417e00640 1 -- 172.21.15.159:0/3033567157 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+88 (secure 0 0 0) 0x7fd4100a05d0 con 0x7fd420101de0 2024-09-19T07:33:46.328 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-19T07:33:46.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.331+0000 7fd425c00640 1 -- 172.21.15.159:0/3033567157 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd4040a3240 msgr2=0x7fd4040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:46.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.331+0000 7fd425c00640 1 --2- 172.21.15.159:0/3033567157 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd4040a3240 0x7fd4040a5700 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd420075f80 tx=0x7fd408005f90 comp rx=0 tx=0).stop 2024-09-19T07:33:46.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.331+0000 7fd425c00640 1 -- 172.21.15.159:0/3033567157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd420101de0 msgr2=0x7fd4200744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:46.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.331+0000 7fd425c00640 1 --2- 172.21.15.159:0/3033567157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd420101de0 0x7fd4200744f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd41000e9f0 tx=0x7fd41000eeb0 comp rx=0 tx=0).stop 2024-09-19T07:33:46.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.331+0000 7fd425c00640 1 -- 172.21.15.159:0/3033567157 shutdown_connections 2024-09-19T07:33:46.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.331+0000 7fd425c00640 1 --2- 172.21.15.159:0/3033567157 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fd4040a3240 0x7fd4040a5700 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:46.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.331+0000 7fd425c00640 1 --2- 172.21.15.159:0/3033567157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd420106f10 0x7fd420074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:46.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.331+0000 7fd425c00640 1 --2- 172.21.15.159:0/3033567157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd420102790 0x7fd420074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:46.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.331+0000 7fd425c00640 1 --2- 172.21.15.159:0/3033567157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd420101de0 0x7fd4200744f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:46.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.331+0000 7fd425c00640 1 -- 172.21.15.159:0/3033567157 >> 172.21.15.159:0/3033567157 conn(0x7fd4200fd8c0 msgr2=0x7fd4200feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:46.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.331+0000 7fd425c00640 1 -- 172.21.15.159:0/3033567157 shutdown_connections 2024-09-19T07:33:46.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:46.331+0000 7fd425c00640 1 -- 172.21.15.159:0/3033567157 wait complete. 2024-09-19T07:33:47.108 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:46 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3033567157' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T07:33:47.147 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:33:47.147 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:33:47.147 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-09-19T07:33:47.147 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:33:47.147 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T07:33:47.147 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:33:47.147 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:33:47.147 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:33:47.148 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:33:47.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:33:47.148 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-09-19T07:33:47.184 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:33:47.184 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:33:47.184 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:33:47.184 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:33:47.184 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-09-19T07:33:47.184 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:33:47.184 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:33:47.184 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:33:47.184 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-09-19T07:33:47.184 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:33:47.185 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:33:47.185 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-09-19T07:33:47.185 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:33:47.185 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-09-19T07:33:47.185 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:33:47.185 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-09-19T07:33:47.185 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T07:33:47.185 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 42021sec preferred_lft 42021sec 2024-09-19T07:33:47.185 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link noprefixroute 2024-09-19T07:33:47.185 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:33:47.186 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T07:33:47.186 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:33:47.187 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-09-19T07:33:47.187 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link add name ceph-brx type bridge 2024-09-19T07:33:47.187 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip addr flush dev ceph-brx 2024-09-19T07:33:47.187 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set ceph-brx up 2024-09-19T07:33:47.187 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T07:33:47.187 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-09-19T07:33:47.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:46 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3033567157' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T07:33:47.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:33:47.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:46 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3033567157' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T07:33:47.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:33:47.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:33:47.415 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:33:47.470 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-09-19T07:33:47.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:33:47.474 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-09-19T07:33:47.510 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-09-19T07:33:47.511 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:33:47.511 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T07:33:47.511 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T07:33:47.511 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:33:47.511 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:33:47.511 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-09-19T07:33:47.511 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T07:33:47.511 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T07:33:47.511 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T07:33:47.511 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-09-19T07:33:47.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:33:47.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:33:47.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:33:47.709 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-09-19T07:33:47.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:33:47.745 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list-id 2024-09-19T07:33:47.823 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:33:47.823 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-09-19T07:33:47.823 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:33:47.823 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T07:33:47.823 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-09-19T07:33:47.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:33:47.958 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:47 smithi159 ceph-mon[30472]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T07:33:47.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:33:48.002 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T07:33:48.002 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:33:48.002 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-09-19T07:33:48.002 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T07:33:48.003 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T07:33:48.003 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T07:33:48.003 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T07:33:48.003 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T07:33:48.003 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-09-19T07:33:48.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:33:48.187 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:47 smithi203 ceph-mon[35335]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T07:33:48.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:33:48.225 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:33:48.225 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-09-19T07:33:48.225 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-09-19T07:33:48.225 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T07:33:48.225 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-09-19T07:33:48.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:33:48.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:47 smithi189 ceph-mon[34851]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-19T07:33:48.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:33:48.391 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:33:48.392 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T07:33:48.427 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T07:33:48.427 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:33:48.427 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:33:48.501 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-19T07:33:48.501 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-19T07:33:48.502 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/7962f356-4b90-4a53-adf1-2e770a41cab4 2024-09-19T07:33:48.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:33:48.502 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/7962f356-4b90-4a53-adf1-2e770a41cab4 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-19T07:33:48.632 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-19T07:33:48.632 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-19T07:33:49.252 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-19T07:33:49.252 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-19T07:33:49.252 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.189:6789,172.21.15.203:6789" 2024-09-19T07:33:49.252 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with new device syntax: 0@c9d18048-7657-11ef-bceb-c7b262605968.cephfs=/volumes/qa/sv_1/7962f356-4b90-4a53-adf1-2e770a41cab4 2024-09-19T07:33:49.252 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.159:6789/172.21.15.189:6789/172.21.15.203:6789" will pass to kernel 2024-09-19T07:33:49.254 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-19T07:33:49.254 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-09-19T07:33:49.254 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-19T07:33:49.254 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-09-19T07:33:49.254 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-19T07:33:49.257 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-19T07:33:49.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:33:49.258 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T07:33:49.316 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T07:33:49.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T07:33:49.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:33:49.408 INFO:teuthology.orchestra.run.smithi159.stdout:client15108 2024-09-19T07:33:49.409 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:33:49.409 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:33:49.447 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T07:33:49.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T07:33:49.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:33:49.577 INFO:teuthology.orchestra.run.smithi159.stdout:client15108 2024-09-19T07:33:49.577 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T07:33:49.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T07:33:49.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:33:49.667 INFO:teuthology.orchestra.run.smithi159.stdout:client15108 2024-09-19T07:33:49.667 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c9d18048-7657-11ef-bceb-c7b262605968.client15108/status') 2024-09-19T07:33:49.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:33:49.768 INFO:teuthology.orchestra.run.smithi159.stdout:instance: client.15108 (3)192.168.0.1:0/772049166 2024-09-19T07:33:49.769 INFO:teuthology.orchestra.run.smithi159.stdout:blocklisted: false 2024-09-19T07:33:49.769 INFO:teuthology.orchestra.run.smithi159.stderr:0+1 records in 2024-09-19T07:33:49.769 INFO:teuthology.orchestra.run.smithi159.stderr:0+1 records out 2024-09-19T07:33:49.769 INFO:teuthology.orchestra.run.smithi159.stderr:69 bytes copied, 5.4644e-05 s, 1.3 MB/s 2024-09-19T07:33:49.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:33:49.778 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T07:33:49.787 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T07:33:49.788 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi159.front.sepia.ceph.com 2024-09-19T07:33:49.788 DEBUG:teuthology.orchestra.run.smithi159:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-19T07:33:49.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.942+0000 7f3dd3200640 1 -- 172.21.15.159:0/2823810913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dcc103d60 msgr2=0x7f3dcc104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:49.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.942+0000 7f3dd3200640 1 --2- 172.21.15.159:0/2823810913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dcc103d60 0x7f3dcc104160 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009900 tx=0x7f3dbc02a3f0 comp rx=0 tx=0).stop 2024-09-19T07:33:49.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.942+0000 7f3dd3200640 1 -- 172.21.15.159:0/2823810913 shutdown_connections 2024-09-19T07:33:49.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.942+0000 7f3dd3200640 1 --2- 172.21.15.159:0/2823810913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dcc105910 0x7f3dcc10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:49.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.942+0000 7f3dd3200640 1 --2- 172.21.15.159:0/2823810913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dcc104f50 0x7f3dcc1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:49.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.942+0000 7f3dd3200640 1 --2- 172.21.15.159:0/2823810913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dcc103d60 0x7f3dcc104160 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:49.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.942+0000 7f3dd3200640 1 -- 172.21.15.159:0/2823810913 >> 172.21.15.159:0/2823810913 conn(0x7f3dcc0ff4e0 msgr2=0x7f3dcc101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:49.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.942+0000 7f3dd3200640 1 -- 172.21.15.159:0/2823810913 shutdown_connections 2024-09-19T07:33:49.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.942+0000 7f3dd3200640 1 -- 172.21.15.159:0/2823810913 wait complete. 2024-09-19T07:33:49.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.943+0000 7f3dd3200640 1 Processor -- start 2024-09-19T07:33:49.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.943+0000 7f3dd3200640 1 -- start start 2024-09-19T07:33:49.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.943+0000 7f3dd3200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dcc103d60 0x7f3dcc199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:49.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.944+0000 7f3dd3200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dcc104f50 0x7f3dcc19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:49.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.944+0000 7f3dd3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dcc105910 0x7f3dcc1a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:49.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.944+0000 7f3dd3200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc10f9c0 con 0x7f3dcc105910 2024-09-19T07:33:49.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.944+0000 7f3dd3200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc10f840 con 0x7f3dcc104f50 2024-09-19T07:33:49.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.944+0000 7f3dd3200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc10fb40 con 0x7f3dcc103d60 2024-09-19T07:33:49.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.944+0000 7f3dd1200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dcc105910 0x7f3dcc1a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:49.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.944+0000 7f3dc3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dcc104f50 0x7f3dcc19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:49.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.944+0000 7f3dd1200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dcc105910 0x7f3dcc1a1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47766/0 (socket says 172.21.15.159:47766) 2024-09-19T07:33:49.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.944+0000 7f3dd1200640 1 -- 172.21.15.159:0/3057066246 learned_addr learned my addr 172.21.15.159:0/3057066246 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:33:49.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.944+0000 7f3dc3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dcc104f50 0x7f3dcc19a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:34164/0 (socket says 172.21.15.159:34164) 2024-09-19T07:33:49.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.944+0000 7f3dcbe00640 1 --2- 172.21.15.159:0/3057066246 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dcc103d60 0x7f3dcc199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:49.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.944+0000 7f3dd1200640 1 -- 172.21.15.159:0/3057066246 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dcc103d60 msgr2=0x7f3dcc199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:49.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.944+0000 7f3dd1200640 1 --2- 172.21.15.159:0/3057066246 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dcc103d60 0x7f3dcc199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:49.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.944+0000 7f3dd1200640 1 -- 172.21.15.159:0/3057066246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dcc104f50 msgr2=0x7f3dcc19a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:49.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.944+0000 7f3dd1200640 1 --2- 172.21.15.159:0/3057066246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dcc104f50 0x7f3dcc19a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:49.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.944+0000 7f3dd1200640 1 -- 172.21.15.159:0/3057066246 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc1a1c00 con 0x7f3dcc105910 2024-09-19T07:33:49.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.944+0000 7f3dc3e00640 1 --2- 172.21.15.159:0/3057066246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dcc104f50 0x7f3dcc19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:33:49.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.945+0000 7f3dd1200640 1 --2- 172.21.15.159:0/3057066246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dcc105910 0x7f3dcc1a1590 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400eb10 tx=0x7f3dc400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:49.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.945+0000 7f3dc9600640 1 -- 172.21.15.159:0/3057066246 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4004b20 con 0x7f3dcc105910 2024-09-19T07:33:49.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.945+0000 7f3dc9600640 1 -- 172.21.15.159:0/3057066246 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dc4004cc0 con 0x7f3dcc105910 2024-09-19T07:33:49.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.945+0000 7f3dc9600640 1 -- 172.21.15.159:0/3057066246 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4010760 con 0x7f3dcc105910 2024-09-19T07:33:49.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.945+0000 7f3dd3200640 1 -- 172.21.15.159:0/3057066246 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dcc1a1ef0 con 0x7f3dcc105910 2024-09-19T07:33:49.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.945+0000 7f3dd3200640 1 -- 172.21.15.159:0/3057066246 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dcc1a2360 con 0x7f3dcc105910 2024-09-19T07:33:49.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.946+0000 7f3dd3200640 1 -- 172.21.15.159:0/3057066246 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dcc10b0d0 con 0x7f3dcc105910 2024-09-19T07:33:49.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.948+0000 7f3dc9600640 1 -- 172.21.15.159:0/3057066246 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f3dc4005020 con 0x7f3dcc105910 2024-09-19T07:33:49.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.949+0000 7f3dc9600640 1 --2- 172.21.15.159:0/3057066246 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3db00a32f0 0x7f3db00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:49.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.949+0000 7f3dc9600640 1 -- 172.21.15.159:0/3057066246 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f3dc40d02d0 con 0x7f3dcc105910 2024-09-19T07:33:49.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.950+0000 7f3dcbe00640 1 --2- 172.21.15.159:0/3057066246 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3db00a32f0 0x7f3db00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:49.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.950+0000 7f3dcbe00640 1 --2- 172.21.15.159:0/3057066246 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3db00a32f0 0x7f3db00a57b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc005ab0 tx=0x7f3dbc000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:49.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:49.954+0000 7f3dc9600640 1 -- 172.21.15.159:0/3057066246 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3dc409cbe0 con 0x7f3dcc105910 2024-09-19T07:33:50.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:49 smithi159 ceph-mon[30472]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:33:50.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:50.166+0000 7f3dd3200640 1 -- 172.21.15.159:0/3057066246 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f3dcc062e90 con 0x7f3dcc105910 2024-09-19T07:33:50.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:49 smithi203 ceph-mon[35335]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:33:50.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:49 smithi189 ceph-mon[34851]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:33:50.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:50.890+0000 7f3dc9600640 1 -- 172.21.15.159:0/3057066246 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v26) ==== 117+0+0 (secure 0 0 0) 0x7f3dc409c4e0 con 0x7f3dcc105910 2024-09-19T07:33:50.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:50.892+0000 7f3dd3200640 1 -- 172.21.15.159:0/3057066246 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3db00a32f0 msgr2=0x7f3db00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:50.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:50.892+0000 7f3dd3200640 1 --2- 172.21.15.159:0/3057066246 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3db00a32f0 0x7f3db00a57b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc005ab0 tx=0x7f3dbc000bc0 comp rx=0 tx=0).stop 2024-09-19T07:33:50.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:50.892+0000 7f3dd3200640 1 -- 172.21.15.159:0/3057066246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dcc105910 msgr2=0x7f3dcc1a1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:50.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:50.892+0000 7f3dd3200640 1 --2- 172.21.15.159:0/3057066246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dcc105910 0x7f3dcc1a1590 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400eb10 tx=0x7f3dc400c2d0 comp rx=0 tx=0).stop 2024-09-19T07:33:50.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:50.892+0000 7f3dd3200640 1 -- 172.21.15.159:0/3057066246 shutdown_connections 2024-09-19T07:33:50.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:50.892+0000 7f3dd3200640 1 --2- 172.21.15.159:0/3057066246 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f3db00a32f0 0x7f3db00a57b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:50.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:50.892+0000 7f3dd3200640 1 --2- 172.21.15.159:0/3057066246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3dcc105910 0x7f3dcc1a1590 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:50.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:50.892+0000 7f3dd3200640 1 --2- 172.21.15.159:0/3057066246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3dcc104f50 0x7f3dcc19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:50.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:50.892+0000 7f3dd3200640 1 --2- 172.21.15.159:0/3057066246 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3dcc103d60 0x7f3dcc199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:50.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:50.892+0000 7f3dd3200640 1 -- 172.21.15.159:0/3057066246 >> 172.21.15.159:0/3057066246 conn(0x7f3dcc0ff4e0 msgr2=0x7f3dcc1082a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:50.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:50.892+0000 7f3dd3200640 1 -- 172.21.15.159:0/3057066246 shutdown_connections 2024-09-19T07:33:50.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:33:50.893+0000 7f3dd3200640 1 -- 172.21.15.159:0/3057066246 wait complete. 2024-09-19T07:33:50.911 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-19T07:33:50.922 INFO:teuthology.run_tasks:Running task workunit... 2024-09-19T07:33:50.958 INFO:tasks.workunit:Pulling workunits from ref 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T07:33:50.959 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-19T07:33:50.959 INFO:tasks.workunit:timeout=3h 2024-09-19T07:33:50.959 INFO:tasks.workunit:cleanup=True 2024-09-19T07:33:50.959 DEBUG:teuthology.orchestra.run.smithi159:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-19T07:33:50.995 INFO:teuthology.orchestra.run.smithi159.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-19T07:33:50.996 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-19T07:33:50.996 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-19T07:33:50.996 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:33:50.996 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-19T07:33:50.996 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-19 07:32:04.031992709 +0000 2024-09-19T07:33:50.996 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-19 07:32:04.031992709 +0000 2024-09-19T07:33:50.996 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-19 07:33:49.311854526 +0000 2024-09-19T07:33:50.996 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-19 07:32:04.031992709 +0000 2024-09-19T07:33:50.996 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-19T07:33:50.996 DEBUG:teuthology.orchestra.run.smithi159:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-19T07:33:51.102 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T07:33:51.140 INFO:tasks.workunit.client.0.smithi159.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-19T07:33:51.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:50 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3057066246' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T07:33:51.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:50 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3057066246' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T07:33:51.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:50 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3057066246' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T07:33:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:51 smithi159 ceph-mon[30472]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T07:33:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:51 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3057066246' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T07:33:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:51 smithi159 ceph-mon[30472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T07:33:52.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:51 smithi203 ceph-mon[35335]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T07:33:52.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:51 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3057066246' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T07:33:52.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:51 smithi203 ceph-mon[35335]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T07:33:52.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:51 smithi189 ceph-mon[34851]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T07:33:52.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:51 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3057066246' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T07:33:52.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:51 smithi189 ceph-mon[34851]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T07:33:54.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:53 smithi203 ceph-mon[35335]: pgmap v401: 129 pgs: 129 active+clean; 648 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T07:33:54.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:53 smithi189 ceph-mon[34851]: pgmap v401: 129 pgs: 129 active+clean; 648 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T07:33:54.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:53 smithi159 ceph-mon[30472]: pgmap v401: 129 pgs: 129 active+clean; 648 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-19T07:33:56.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:55 smithi203 ceph-mon[35335]: pgmap v402: 129 pgs: 129 active+clean; 648 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-19T07:33:56.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:55 smithi189 ceph-mon[34851]: pgmap v402: 129 pgs: 129 active+clean; 648 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-19T07:33:56.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:55 smithi159 ceph-mon[30472]: pgmap v402: 129 pgs: 129 active+clean; 648 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-19T07:33:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:57 smithi203 ceph-mon[35335]: pgmap v403: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-19T07:33:58.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:57 smithi189 ceph-mon[34851]: pgmap v403: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-19T07:33:58.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:57 smithi159 ceph-mon[30472]: pgmap v403: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-19T07:34:00.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:33:59 smithi203 ceph-mon[35335]: pgmap v404: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-19T07:34:00.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:33:59 smithi189 ceph-mon[34851]: pgmap v404: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-19T07:34:00.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:33:59 smithi159 ceph-mon[30472]: pgmap v404: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-19T07:34:02.162 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:01 smithi159 ceph-mon[30472]: pgmap v405: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-19T07:34:02.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:01 smithi203 ceph-mon[35335]: pgmap v405: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-19T07:34:02.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:01 smithi189 ceph-mon[34851]: pgmap v405: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-19T07:34:04.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:03 smithi203 ceph-mon[35335]: pgmap v406: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-09-19T07:34:04.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:03 smithi189 ceph-mon[34851]: pgmap v406: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-09-19T07:34:04.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:03 smithi159 ceph-mon[30472]: pgmap v406: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-09-19T07:34:06.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:05 smithi203 ceph-mon[35335]: pgmap v407: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T07:34:06.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:05 smithi189 ceph-mon[34851]: pgmap v407: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T07:34:06.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:05 smithi159 ceph-mon[30472]: pgmap v407: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T07:34:08.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:07 smithi203 ceph-mon[35335]: pgmap v408: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-19T07:34:08.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:07 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:34:08.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:07 smithi189 ceph-mon[34851]: pgmap v408: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-19T07:34:08.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:07 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:34:08.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:07 smithi159 ceph-mon[30472]: pgmap v408: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-19T07:34:08.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:07 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:34:09.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:08 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:08 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:08 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:08 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:08 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:08 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:08 smithi203 ceph-mon[35335]: pgmap v409: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:34:09.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:08 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:09.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:08 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:09.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:08 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:08 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:08 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:08 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:08 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:08 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:08 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:08 smithi189 ceph-mon[34851]: pgmap v409: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:34:09.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:08 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:09.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:08 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:09.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:08 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:08 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:08 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:08 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:08 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:08 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:08 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:09.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:08 smithi159 ceph-mon[30472]: pgmap v409: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:34:09.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:08 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:09.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:08 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:09.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:08 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:34:12.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:11 smithi189 ceph-mon[34851]: pgmap v410: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:12.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:11 smithi159 ceph-mon[30472]: pgmap v410: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:12.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:11 smithi203 ceph-mon[35335]: pgmap v410: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:14.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:13 smithi189 ceph-mon[34851]: pgmap v411: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:34:14.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:13 smithi159 ceph-mon[30472]: pgmap v411: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:34:14.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:13 smithi203 ceph-mon[35335]: pgmap v411: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:34:16.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:15 smithi189 ceph-mon[34851]: pgmap v412: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:16.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:15 smithi159 ceph-mon[30472]: pgmap v412: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:16.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:15 smithi203 ceph-mon[35335]: pgmap v412: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:18.088 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:17 smithi203 ceph-mon[35335]: pgmap v413: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:34:18.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:17 smithi159 ceph-mon[30472]: pgmap v413: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:34:18.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:17 smithi189 ceph-mon[34851]: pgmap v413: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:34:20.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:19 smithi159 ceph-mon[30472]: pgmap v414: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:34:20.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:19 smithi203 ceph-mon[35335]: pgmap v414: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:34:20.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:19 smithi189 ceph-mon[34851]: pgmap v414: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:34:22.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:21 smithi159 ceph-mon[30472]: pgmap v415: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:22.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:21 smithi203 ceph-mon[35335]: pgmap v415: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:22.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:21 smithi189 ceph-mon[34851]: pgmap v415: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:24.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:23 smithi159 ceph-mon[30472]: pgmap v416: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:34:24.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:23 smithi203 ceph-mon[35335]: pgmap v416: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:34:24.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:23 smithi189 ceph-mon[34851]: pgmap v416: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:34:26.108 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:25 smithi189 ceph-mon[34851]: pgmap v417: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:26.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:25 smithi159 ceph-mon[30472]: pgmap v417: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:26.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:25 smithi203 ceph-mon[35335]: pgmap v417: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:27 smithi159 ceph-mon[30472]: pgmap v418: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:34:28.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:27 smithi203 ceph-mon[35335]: pgmap v418: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:34:28.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:27 smithi189 ceph-mon[34851]: pgmap v418: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:34:30.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:29 smithi159 ceph-mon[30472]: pgmap v419: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:34:30.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:29 smithi203 ceph-mon[35335]: pgmap v419: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:34:30.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:29 smithi189 ceph-mon[34851]: pgmap v419: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:34:32.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:31 smithi159 ceph-mon[30472]: pgmap v420: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T07:34:32.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:31 smithi203 ceph-mon[35335]: pgmap v420: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T07:34:32.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:31 smithi189 ceph-mon[34851]: pgmap v420: 129 pgs: 129 active+clean; 653 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T07:34:34.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:33 smithi159 ceph-mon[30472]: pgmap v421: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-19T07:34:34.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:33 smithi203 ceph-mon[35335]: pgmap v421: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-19T07:34:34.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:33 smithi189 ceph-mon[34851]: pgmap v421: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-19T07:34:36.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:35 smithi159 ceph-mon[30472]: pgmap v422: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T07:34:36.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:35 smithi203 ceph-mon[35335]: pgmap v422: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T07:34:36.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:35 smithi189 ceph-mon[34851]: pgmap v422: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T07:34:38.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:37 smithi159 ceph-mon[30472]: pgmap v423: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-19T07:34:38.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:37 smithi203 ceph-mon[35335]: pgmap v423: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-19T07:34:38.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:37 smithi189 ceph-mon[34851]: pgmap v423: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-19T07:34:40.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:39 smithi159 ceph-mon[30472]: pgmap v424: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T07:34:40.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:39 smithi203 ceph-mon[35335]: pgmap v424: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T07:34:40.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:39 smithi189 ceph-mon[34851]: pgmap v424: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T07:34:42.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:41 smithi159 ceph-mon[30472]: pgmap v425: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T07:34:42.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:41 smithi203 ceph-mon[35335]: pgmap v425: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T07:34:42.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:41 smithi189 ceph-mon[34851]: pgmap v425: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T07:34:44.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:43 smithi159 ceph-mon[30472]: pgmap v426: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-19T07:34:44.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:43 smithi203 ceph-mon[35335]: pgmap v426: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-19T07:34:44.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:43 smithi189 ceph-mon[34851]: pgmap v426: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-19T07:34:46.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:45 smithi159 ceph-mon[30472]: pgmap v427: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:46.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:45 smithi203 ceph-mon[35335]: pgmap v427: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:46.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:45 smithi189 ceph-mon[34851]: pgmap v427: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:48.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:47 smithi159 ceph-mon[30472]: pgmap v428: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:34:48.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:47 smithi203 ceph-mon[35335]: pgmap v428: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:34:48.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:47 smithi189 ceph-mon[34851]: pgmap v428: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-19T07:34:50.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:49 smithi159 ceph-mon[30472]: pgmap v429: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:34:50.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:49 smithi203 ceph-mon[35335]: pgmap v429: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:34:50.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:49 smithi189 ceph-mon[34851]: pgmap v429: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T07:34:52.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:51 smithi159 ceph-mon[30472]: pgmap v430: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:52.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:51 smithi203 ceph-mon[35335]: pgmap v430: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:52.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:51 smithi189 ceph-mon[34851]: pgmap v430: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:52.889 INFO:tasks.workunit.client.0.smithi159.stderr:Updating files: 64% (8333/12896) Updating files: 65% (8383/12896) Updating files: 66% (8512/12896) Updating files: 67% (8641/12896) Updating files: 68% (8770/12896) Updating files: 69% (8899/12896) Updating files: 70% (9028/12896) Updating files: 71% (9157/12896) Updating files: 72% (9286/12896) Updating files: 73% (9415/12896) Updating files: 74% (9544/12896) Updating files: 75% (9672/12896) Updating files: 76% (9801/12896) Updating files: 77% (9930/12896) Updating files: 78% (10059/12896) Updating files: 79% (10188/12896) Updating files: 80% (10317/12896) Updating files: 81% (10446/12896) Updating files: 82% (10575/12896) Updating files: 83% (10704/12896) Updating files: 84% (10833/12896) Updating files: 85% (10962/12896) Updating files: 86% (11091/12896) Updating files: 87% (11220/12896) Updating files: 88% (11349/12896) Updating files: 89% (11478/12896) Updating files: 90% (11607/12896) Updating files: 91% (11736/12896) Updating files: 92% (11865/12896) Updating files: 93% (11994/12896) Updating files: 94% (12123/12896) Updating files: 95% (12252/12896) Updating files: 96% (12381/12896) Updating files: 97% (12510/12896) Updating files: 98% (12639/12896) Updating files: 99% (12768/12896) Updating files: 100% (12896/12896) Updating files: 100% (12896/12896), done. 2024-09-19T07:34:53.971 INFO:tasks.workunit.client.0.smithi159.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T07:34:53.971 INFO:tasks.workunit.client.0.smithi159.stderr: 2024-09-19T07:34:53.971 INFO:tasks.workunit.client.0.smithi159.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T07:34:53.971 INFO:tasks.workunit.client.0.smithi159.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T07:34:53.971 INFO:tasks.workunit.client.0.smithi159.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T07:34:53.971 INFO:tasks.workunit.client.0.smithi159.stderr: 2024-09-19T07:34:53.971 INFO:tasks.workunit.client.0.smithi159.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T07:34:53.971 INFO:tasks.workunit.client.0.smithi159.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T07:34:53.971 INFO:tasks.workunit.client.0.smithi159.stderr: 2024-09-19T07:34:53.972 INFO:tasks.workunit.client.0.smithi159.stderr: git switch -c 2024-09-19T07:34:53.972 INFO:tasks.workunit.client.0.smithi159.stderr: 2024-09-19T07:34:53.972 INFO:tasks.workunit.client.0.smithi159.stderr:Or undo this operation with: 2024-09-19T07:34:53.972 INFO:tasks.workunit.client.0.smithi159.stderr: 2024-09-19T07:34:53.972 INFO:tasks.workunit.client.0.smithi159.stderr: git switch - 2024-09-19T07:34:53.972 INFO:tasks.workunit.client.0.smithi159.stderr: 2024-09-19T07:34:53.972 INFO:tasks.workunit.client.0.smithi159.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T07:34:53.972 INFO:tasks.workunit.client.0.smithi159.stderr: 2024-09-19T07:34:53.972 INFO:tasks.workunit.client.0.smithi159.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T07:34:53.986 DEBUG:teuthology.orchestra.run.smithi159:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-19T07:34:54.075 INFO:tasks.workunit.client.0.smithi159.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T07:34:54.080 INFO:tasks.workunit.client.0.smithi159.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T07:34:54.080 INFO:tasks.workunit.client.0.smithi159.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T07:34:54.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:53 smithi159 ceph-mon[30472]: pgmap v431: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:34:54.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:53 smithi203 ceph-mon[35335]: pgmap v431: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:34:54.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:53 smithi189 ceph-mon[34851]: pgmap v431: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:34:54.786 INFO:tasks.workunit.client.0.smithi159.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T07:34:54.887 INFO:tasks.workunit.client.0.smithi159.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T07:34:54.953 INFO:tasks.workunit.client.0.smithi159.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T07:34:54.957 INFO:tasks.workunit.client.0.smithi159.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T07:34:54.957 INFO:tasks.workunit.client.0.smithi159.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T07:34:55.023 INFO:tasks.workunit.client.0.smithi159.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T07:34:55.030 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:34:55.030 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-19T07:34:55.066 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-09-19T07:34:55.067 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-09-19T07:34:55.068 DEBUG:teuthology.orchestra.run.smithi159:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-09-19T07:34:55.157 INFO:tasks.workunit.client.0.smithi159.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-19T07:34:55.292 INFO:tasks.workunit.client.0.smithi159.stdout: Iozone: Performance Test of File I/O 2024-09-19T07:34:55.293 INFO:tasks.workunit.client.0.smithi159.stdout: Version $Revision: 3.494 $ 2024-09-19T07:34:55.293 INFO:tasks.workunit.client.0.smithi159.stdout: Compiled for 64 bit mode. 2024-09-19T07:34:55.293 INFO:tasks.workunit.client.0.smithi159.stdout: Build: linux-AMD64 2024-09-19T07:34:55.293 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:34:55.293 INFO:tasks.workunit.client.0.smithi159.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-19T07:34:55.293 INFO:tasks.workunit.client.0.smithi159.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-19T07:34:55.293 INFO:tasks.workunit.client.0.smithi159.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-19T07:34:55.293 INFO:tasks.workunit.client.0.smithi159.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-19T07:34:55.293 INFO:tasks.workunit.client.0.smithi159.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-19T07:34:55.293 INFO:tasks.workunit.client.0.smithi159.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-19T07:34:55.293 INFO:tasks.workunit.client.0.smithi159.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-19T07:34:55.294 INFO:tasks.workunit.client.0.smithi159.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-19T07:34:55.294 INFO:tasks.workunit.client.0.smithi159.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-19T07:34:55.294 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:34:55.294 INFO:tasks.workunit.client.0.smithi159.stdout: Run began: Thu Sep 19 07:34:55 2024 2024-09-19T07:34:55.294 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:34:55.294 INFO:tasks.workunit.client.0.smithi159.stdout: Include close in write timing 2024-09-19T07:34:55.294 INFO:tasks.workunit.client.0.smithi159.stdout: Include fsync in write timing 2024-09-19T07:34:55.294 INFO:tasks.workunit.client.0.smithi159.stdout: File size set to 1048576 kB 2024-09-19T07:34:55.294 INFO:tasks.workunit.client.0.smithi159.stdout: Record Size 16 kB 2024-09-19T07:34:55.294 INFO:tasks.workunit.client.0.smithi159.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-19T07:34:55.294 INFO:tasks.workunit.client.0.smithi159.stdout: Output is in kBytes/sec 2024-09-19T07:34:55.294 INFO:tasks.workunit.client.0.smithi159.stdout: Time Resolution = 0.000001 seconds. 2024-09-19T07:34:55.294 INFO:tasks.workunit.client.0.smithi159.stdout: Processor cache size set to 1024 kBytes. 2024-09-19T07:34:55.294 INFO:tasks.workunit.client.0.smithi159.stdout: Processor cache line size set to 32 bytes. 2024-09-19T07:34:55.295 INFO:tasks.workunit.client.0.smithi159.stdout: File stride size set to 17 * record size. 2024-09-19T07:34:55.295 INFO:tasks.workunit.client.0.smithi159.stdout: Throughput test with 1 process 2024-09-19T07:34:55.295 INFO:tasks.workunit.client.0.smithi159.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-09-19T07:34:56.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:55 smithi159 ceph-mon[30472]: pgmap v432: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:56.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:55 smithi203 ceph-mon[35335]: pgmap v432: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:56.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:55 smithi189 ceph-mon[34851]: pgmap v432: 129 pgs: 129 active+clean; 658 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:34:58.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:57 smithi159 ceph-mon[30472]: pgmap v433: 129 pgs: 129 active+clean; 691 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-19T07:34:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:57 smithi203 ceph-mon[35335]: pgmap v433: 129 pgs: 129 active+clean; 691 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-19T07:34:58.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:57 smithi189 ceph-mon[34851]: pgmap v433: 129 pgs: 129 active+clean; 691 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-19T07:34:59.647 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:34:59.647 INFO:tasks.workunit.client.0.smithi159.stdout: Children see throughput for 1 initial writers = 241014.89 kB/sec 2024-09-19T07:34:59.647 INFO:tasks.workunit.client.0.smithi159.stdout: Parent sees throughput for 1 initial writers = 240977.89 kB/sec 2024-09-19T07:34:59.647 INFO:tasks.workunit.client.0.smithi159.stdout: Min throughput per process = 241014.89 kB/sec 2024-09-19T07:34:59.647 INFO:tasks.workunit.client.0.smithi159.stdout: Max throughput per process = 241014.89 kB/sec 2024-09-19T07:34:59.647 INFO:tasks.workunit.client.0.smithi159.stdout: Avg throughput per process = 241014.89 kB/sec 2024-09-19T07:34:59.648 INFO:tasks.workunit.client.0.smithi159.stdout: Min xfer = 1048576.00 kB 2024-09-19T07:35:00.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:34:59 smithi159 ceph-mon[30472]: pgmap v434: 129 pgs: 129 active+clean; 89 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7.3 MiB/s wr, 8 op/s 2024-09-19T07:35:00.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:34:59 smithi203 ceph-mon[35335]: pgmap v434: 129 pgs: 129 active+clean; 89 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7.3 MiB/s wr, 8 op/s 2024-09-19T07:35:00.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:34:59 smithi189 ceph-mon[34851]: pgmap v434: 129 pgs: 129 active+clean; 89 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7.3 MiB/s wr, 8 op/s 2024-09-19T07:35:02.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:01 smithi159 ceph-mon[30472]: pgmap v435: 129 pgs: 129 active+clean; 389 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 32 MiB/s wr, 14 op/s 2024-09-19T07:35:02.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:01 smithi203 ceph-mon[35335]: pgmap v435: 129 pgs: 129 active+clean; 389 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 32 MiB/s wr, 14 op/s 2024-09-19T07:35:02.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:01 smithi189 ceph-mon[34851]: pgmap v435: 129 pgs: 129 active+clean; 389 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 32 MiB/s wr, 14 op/s 2024-09-19T07:35:04.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:03 smithi203 ceph-mon[35335]: pgmap v436: 129 pgs: 129 active+clean; 913 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 76 MiB/s wr, 28 op/s 2024-09-19T07:35:04.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:03 smithi189 ceph-mon[34851]: pgmap v436: 129 pgs: 129 active+clean; 913 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 76 MiB/s wr, 28 op/s 2024-09-19T07:35:04.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:03 smithi159 ceph-mon[30472]: pgmap v436: 129 pgs: 129 active+clean; 913 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 76 MiB/s wr, 28 op/s 2024-09-19T07:35:06.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:05 smithi203 ceph-mon[35335]: pgmap v437: 129 pgs: 129 active+clean; 1.0 GiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 86 MiB/s wr, 28 op/s 2024-09-19T07:35:06.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:05 smithi189 ceph-mon[34851]: pgmap v437: 129 pgs: 129 active+clean; 1.0 GiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 86 MiB/s wr, 28 op/s 2024-09-19T07:35:06.392 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:05 smithi159 ceph-mon[30472]: pgmap v437: 129 pgs: 129 active+clean; 1.0 GiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 86 MiB/s wr, 28 op/s 2024-09-19T07:35:07.034 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:07.034 INFO:tasks.workunit.client.0.smithi159.stdout: Children see throughput for 1 rewriters = 300283.97 kB/sec 2024-09-19T07:35:07.034 INFO:tasks.workunit.client.0.smithi159.stdout: Parent sees throughput for 1 rewriters = 300231.77 kB/sec 2024-09-19T07:35:07.034 INFO:tasks.workunit.client.0.smithi159.stdout: Min throughput per process = 300283.97 kB/sec 2024-09-19T07:35:07.034 INFO:tasks.workunit.client.0.smithi159.stdout: Max throughput per process = 300283.97 kB/sec 2024-09-19T07:35:07.034 INFO:tasks.workunit.client.0.smithi159.stdout: Avg throughput per process = 300283.97 kB/sec 2024-09-19T07:35:07.034 INFO:tasks.workunit.client.0.smithi159.stdout: Min xfer = 1048576.00 kB 2024-09-19T07:35:08.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:07 smithi203 ceph-mon[35335]: pgmap v438: 129 pgs: 129 active+clean; 1.0 GiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 109 MiB/s wr, 36 op/s 2024-09-19T07:35:08.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:07 smithi189 ceph-mon[34851]: pgmap v438: 129 pgs: 129 active+clean; 1.0 GiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 109 MiB/s wr, 36 op/s 2024-09-19T07:35:08.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:07 smithi159 ceph-mon[30472]: pgmap v438: 129 pgs: 129 active+clean; 1.0 GiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 109 MiB/s wr, 36 op/s 2024-09-19T07:35:10.145 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:10.145 INFO:tasks.workunit.client.0.smithi159.stdout: Children see throughput for 1 readers = 3571348.25 kB/sec 2024-09-19T07:35:10.146 INFO:tasks.workunit.client.0.smithi159.stdout: Parent sees throughput for 1 readers = 3564511.21 kB/sec 2024-09-19T07:35:10.146 INFO:tasks.workunit.client.0.smithi159.stdout: Min throughput per process = 3571348.25 kB/sec 2024-09-19T07:35:10.146 INFO:tasks.workunit.client.0.smithi159.stdout: Max throughput per process = 3571348.25 kB/sec 2024-09-19T07:35:10.146 INFO:tasks.workunit.client.0.smithi159.stdout: Avg throughput per process = 3571348.25 kB/sec 2024-09-19T07:35:10.146 INFO:tasks.workunit.client.0.smithi159.stdout: Min xfer = 1048576.00 kB 2024-09-19T07:35:10.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:09 smithi203 ceph-mon[35335]: pgmap v439: 129 pgs: 129 active+clean; 1.0 GiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 134 MiB/s wr, 39 op/s 2024-09-19T07:35:10.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:09 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:35:10.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:09 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:10.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:09 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:35:10.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:09 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:35:10.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:09 smithi189 ceph-mon[34851]: pgmap v439: 129 pgs: 129 active+clean; 1.0 GiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 134 MiB/s wr, 39 op/s 2024-09-19T07:35:10.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:09 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:35:10.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:09 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:10.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:09 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:35:10.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:09 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:35:10.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:09 smithi159 ceph-mon[30472]: pgmap v439: 129 pgs: 129 active+clean; 1.0 GiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 134 MiB/s wr, 39 op/s 2024-09-19T07:35:10.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:09 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:35:10.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:09 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:10.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:09 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:35:10.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:09 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:35:12.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:11 smithi203 ceph-mon[35335]: pgmap v440: 129 pgs: 129 active+clean; 1.0 GiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 151 MiB/s wr, 43 op/s 2024-09-19T07:35:12.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:11 smithi189 ceph-mon[34851]: pgmap v440: 129 pgs: 129 active+clean; 1.0 GiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 151 MiB/s wr, 43 op/s 2024-09-19T07:35:12.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:11 smithi159 ceph-mon[30472]: pgmap v440: 129 pgs: 129 active+clean; 1.0 GiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 151 MiB/s wr, 43 op/s 2024-09-19T07:35:12.650 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:12.651 INFO:tasks.workunit.client.0.smithi159.stdout: Children see throughput for 1 re-readers = 2696337.50 kB/sec 2024-09-19T07:35:12.651 INFO:tasks.workunit.client.0.smithi159.stdout: Parent sees throughput for 1 re-readers = 2692336.25 kB/sec 2024-09-19T07:35:12.651 INFO:tasks.workunit.client.0.smithi159.stdout: Min throughput per process = 2696337.50 kB/sec 2024-09-19T07:35:12.651 INFO:tasks.workunit.client.0.smithi159.stdout: Max throughput per process = 2696337.50 kB/sec 2024-09-19T07:35:12.651 INFO:tasks.workunit.client.0.smithi159.stdout: Avg throughput per process = 2696337.50 kB/sec 2024-09-19T07:35:12.651 INFO:tasks.workunit.client.0.smithi159.stdout: Min xfer = 1048576.00 kB 2024-09-19T07:35:14.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:13 smithi203 ceph-mon[35335]: pgmap v441: 129 pgs: 129 active+clean; 1.0 GiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 138 MiB/s wr, 42 op/s 2024-09-19T07:35:14.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:13 smithi189 ceph-mon[34851]: pgmap v441: 129 pgs: 129 active+clean; 1.0 GiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 138 MiB/s wr, 42 op/s 2024-09-19T07:35:14.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:13 smithi159 ceph-mon[30472]: pgmap v441: 129 pgs: 129 active+clean; 1.0 GiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 138 MiB/s wr, 42 op/s 2024-09-19T07:35:16.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:15 smithi203 ceph-mon[35335]: pgmap v442: 129 pgs: 129 active+clean; 1.0 GiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 95 MiB/s wr, 29 op/s 2024-09-19T07:35:16.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:15 smithi189 ceph-mon[34851]: pgmap v442: 129 pgs: 129 active+clean; 1.0 GiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 95 MiB/s wr, 29 op/s 2024-09-19T07:35:16.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:15 smithi159 ceph-mon[30472]: pgmap v442: 129 pgs: 129 active+clean; 1.0 GiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 95 MiB/s wr, 29 op/s 2024-09-19T07:35:18.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:17 smithi203 ceph-mon[35335]: pgmap v443: 129 pgs: 129 active+clean; 1.0 GiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 MiB/s wr, 28 op/s 2024-09-19T07:35:18.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:17 smithi189 ceph-mon[34851]: pgmap v443: 129 pgs: 129 active+clean; 1.0 GiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 MiB/s wr, 28 op/s 2024-09-19T07:35:18.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:17 smithi159 ceph-mon[30472]: pgmap v443: 129 pgs: 129 active+clean; 1.0 GiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 MiB/s wr, 28 op/s 2024-09-19T07:35:19.451 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:19.451 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:19.451 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:19.451 INFO:tasks.workunit.client.0.smithi159.stdout:iozone test complete. 2024-09-19T07:35:19.453 INFO:tasks.workunit.client.0.smithi159.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-19T07:35:19.531 INFO:tasks.workunit.client.0.smithi159.stdout: Iozone: Performance Test of File I/O 2024-09-19T07:35:19.532 INFO:tasks.workunit.client.0.smithi159.stdout: Version $Revision: 3.494 $ 2024-09-19T07:35:19.532 INFO:tasks.workunit.client.0.smithi159.stdout: Compiled for 64 bit mode. 2024-09-19T07:35:19.532 INFO:tasks.workunit.client.0.smithi159.stdout: Build: linux-AMD64 2024-09-19T07:35:19.532 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:19.532 INFO:tasks.workunit.client.0.smithi159.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-19T07:35:19.532 INFO:tasks.workunit.client.0.smithi159.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-19T07:35:19.532 INFO:tasks.workunit.client.0.smithi159.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-19T07:35:19.532 INFO:tasks.workunit.client.0.smithi159.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-19T07:35:19.532 INFO:tasks.workunit.client.0.smithi159.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-19T07:35:19.532 INFO:tasks.workunit.client.0.smithi159.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-19T07:35:19.532 INFO:tasks.workunit.client.0.smithi159.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-19T07:35:19.532 INFO:tasks.workunit.client.0.smithi159.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-19T07:35:19.532 INFO:tasks.workunit.client.0.smithi159.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-19T07:35:19.533 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:19.533 INFO:tasks.workunit.client.0.smithi159.stdout: Run began: Thu Sep 19 07:35:19 2024 2024-09-19T07:35:19.533 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:19.533 INFO:tasks.workunit.client.0.smithi159.stdout: Include close in write timing 2024-09-19T07:35:19.533 INFO:tasks.workunit.client.0.smithi159.stdout: Include fsync in write timing 2024-09-19T07:35:19.533 INFO:tasks.workunit.client.0.smithi159.stdout: File size set to 1048576 kB 2024-09-19T07:35:19.533 INFO:tasks.workunit.client.0.smithi159.stdout: Record Size 1024 kB 2024-09-19T07:35:19.533 INFO:tasks.workunit.client.0.smithi159.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-19T07:35:19.533 INFO:tasks.workunit.client.0.smithi159.stdout: Output is in kBytes/sec 2024-09-19T07:35:19.533 INFO:tasks.workunit.client.0.smithi159.stdout: Time Resolution = 0.000001 seconds. 2024-09-19T07:35:19.533 INFO:tasks.workunit.client.0.smithi159.stdout: Processor cache size set to 1024 kBytes. 2024-09-19T07:35:19.533 INFO:tasks.workunit.client.0.smithi159.stdout: Processor cache line size set to 32 bytes. 2024-09-19T07:35:19.533 INFO:tasks.workunit.client.0.smithi159.stdout: File stride size set to 17 * record size. 2024-09-19T07:35:19.533 INFO:tasks.workunit.client.0.smithi159.stdout: Throughput test with 1 process 2024-09-19T07:35:19.534 INFO:tasks.workunit.client.0.smithi159.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-09-19T07:35:20.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:19 smithi203 ceph-mon[35335]: pgmap v444: 129 pgs: 129 active+clean; 905 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 61 MiB/s wr, 23 op/s 2024-09-19T07:35:20.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:19 smithi189 ceph-mon[34851]: pgmap v444: 129 pgs: 129 active+clean; 905 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 61 MiB/s wr, 23 op/s 2024-09-19T07:35:20.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:19 smithi159 ceph-mon[30472]: pgmap v444: 129 pgs: 129 active+clean; 905 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 61 MiB/s wr, 23 op/s 2024-09-19T07:35:22.191 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:21 smithi159 ceph-mon[30472]: pgmap v445: 129 pgs: 129 active+clean; 569 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 36 MiB/s wr, 23 op/s 2024-09-19T07:35:22.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:21 smithi203 ceph-mon[35335]: pgmap v445: 129 pgs: 129 active+clean; 569 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 36 MiB/s wr, 23 op/s 2024-09-19T07:35:22.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:21 smithi189 ceph-mon[34851]: pgmap v445: 129 pgs: 129 active+clean; 569 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 36 MiB/s wr, 23 op/s 2024-09-19T07:35:23.454 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:23.455 INFO:tasks.workunit.client.0.smithi159.stdout: Children see throughput for 1 initial writers = 269644.28 kB/sec 2024-09-19T07:35:23.455 INFO:tasks.workunit.client.0.smithi159.stdout: Parent sees throughput for 1 initial writers = 269596.80 kB/sec 2024-09-19T07:35:23.455 INFO:tasks.workunit.client.0.smithi159.stdout: Min throughput per process = 269644.28 kB/sec 2024-09-19T07:35:23.455 INFO:tasks.workunit.client.0.smithi159.stdout: Max throughput per process = 269644.28 kB/sec 2024-09-19T07:35:23.455 INFO:tasks.workunit.client.0.smithi159.stdout: Avg throughput per process = 269644.28 kB/sec 2024-09-19T07:35:23.455 INFO:tasks.workunit.client.0.smithi159.stdout: Min xfer = 1048576.00 kB 2024-09-19T07:35:24.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:23 smithi203 ceph-mon[35335]: pgmap v446: 129 pgs: 129 active+clean; 189 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 28 MiB/s wr, 37 op/s 2024-09-19T07:35:24.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:23 smithi189 ceph-mon[34851]: pgmap v446: 129 pgs: 129 active+clean; 189 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 28 MiB/s wr, 37 op/s 2024-09-19T07:35:24.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:23 smithi159 ceph-mon[30472]: pgmap v446: 129 pgs: 129 active+clean; 189 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 28 MiB/s wr, 37 op/s 2024-09-19T07:35:26.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:25 smithi203 ceph-mon[35335]: pgmap v447: 129 pgs: 129 active+clean; 549 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 46 MiB/s wr, 39 op/s 2024-09-19T07:35:26.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:25 smithi189 ceph-mon[34851]: pgmap v447: 129 pgs: 129 active+clean; 549 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 46 MiB/s wr, 39 op/s 2024-09-19T07:35:26.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:25 smithi159 ceph-mon[30472]: pgmap v447: 129 pgs: 129 active+clean; 549 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 46 MiB/s wr, 39 op/s 2024-09-19T07:35:28.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:27 smithi203 ceph-mon[35335]: pgmap v448: 129 pgs: 129 active+clean; 945 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 79 MiB/s wr, 50 op/s 2024-09-19T07:35:28.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:27 smithi189 ceph-mon[34851]: pgmap v448: 129 pgs: 129 active+clean; 945 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 79 MiB/s wr, 50 op/s 2024-09-19T07:35:28.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:27 smithi159 ceph-mon[30472]: pgmap v448: 129 pgs: 129 active+clean; 945 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 79 MiB/s wr, 50 op/s 2024-09-19T07:35:29.874 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:29.875 INFO:tasks.workunit.client.0.smithi159.stdout: Children see throughput for 1 rewriters = 260816.66 kB/sec 2024-09-19T07:35:29.875 INFO:tasks.workunit.client.0.smithi159.stdout: Parent sees throughput for 1 rewriters = 260772.59 kB/sec 2024-09-19T07:35:29.875 INFO:tasks.workunit.client.0.smithi159.stdout: Min throughput per process = 260816.66 kB/sec 2024-09-19T07:35:29.875 INFO:tasks.workunit.client.0.smithi159.stdout: Max throughput per process = 260816.66 kB/sec 2024-09-19T07:35:29.875 INFO:tasks.workunit.client.0.smithi159.stdout: Avg throughput per process = 260816.66 kB/sec 2024-09-19T07:35:29.875 INFO:tasks.workunit.client.0.smithi159.stdout: Min xfer = 1048576.00 kB 2024-09-19T07:35:30.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:29 smithi203 ceph-mon[35335]: pgmap v449: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 95 MiB/s wr, 53 op/s 2024-09-19T07:35:30.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:29 smithi189 ceph-mon[34851]: pgmap v449: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 95 MiB/s wr, 53 op/s 2024-09-19T07:35:30.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:29 smithi159 ceph-mon[30472]: pgmap v449: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 95 MiB/s wr, 53 op/s 2024-09-19T07:35:32.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:31 smithi203 ceph-mon[35335]: pgmap v450: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 117 MiB/s wr, 55 op/s 2024-09-19T07:35:32.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:31 smithi189 ceph-mon[34851]: pgmap v450: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 117 MiB/s wr, 55 op/s 2024-09-19T07:35:32.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:31 smithi159 ceph-mon[30472]: pgmap v450: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 117 MiB/s wr, 55 op/s 2024-09-19T07:35:32.907 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:32.907 INFO:tasks.workunit.client.0.smithi159.stdout: Children see throughput for 1 readers = 4789352.00 kB/sec 2024-09-19T07:35:32.907 INFO:tasks.workunit.client.0.smithi159.stdout: Parent sees throughput for 1 readers = 4774041.39 kB/sec 2024-09-19T07:35:32.907 INFO:tasks.workunit.client.0.smithi159.stdout: Min throughput per process = 4789352.00 kB/sec 2024-09-19T07:35:32.907 INFO:tasks.workunit.client.0.smithi159.stdout: Max throughput per process = 4789352.00 kB/sec 2024-09-19T07:35:32.907 INFO:tasks.workunit.client.0.smithi159.stdout: Avg throughput per process = 4789352.00 kB/sec 2024-09-19T07:35:32.907 INFO:tasks.workunit.client.0.smithi159.stdout: Min xfer = 1048576.00 kB 2024-09-19T07:35:34.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:33 smithi203 ceph-mon[35335]: pgmap v451: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 159 MiB/s wr, 62 op/s 2024-09-19T07:35:34.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:33 smithi189 ceph-mon[34851]: pgmap v451: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 159 MiB/s wr, 62 op/s 2024-09-19T07:35:34.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:33 smithi159 ceph-mon[30472]: pgmap v451: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 159 MiB/s wr, 62 op/s 2024-09-19T07:35:35.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:35 smithi203 ceph-mon[35335]: pgmap v452: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 155 MiB/s wr, 44 op/s 2024-09-19T07:35:35.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:35 smithi189 ceph-mon[34851]: pgmap v452: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 155 MiB/s wr, 44 op/s 2024-09-19T07:35:35.313 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:35.313 INFO:tasks.workunit.client.0.smithi159.stdout: Children see throughput for 1 re-readers = 3459722.25 kB/sec 2024-09-19T07:35:35.313 INFO:tasks.workunit.client.0.smithi159.stdout: Parent sees throughput for 1 re-readers = 3451885.40 kB/sec 2024-09-19T07:35:35.314 INFO:tasks.workunit.client.0.smithi159.stdout: Min throughput per process = 3459722.25 kB/sec 2024-09-19T07:35:35.314 INFO:tasks.workunit.client.0.smithi159.stdout: Max throughput per process = 3459722.25 kB/sec 2024-09-19T07:35:35.314 INFO:tasks.workunit.client.0.smithi159.stdout: Avg throughput per process = 3459722.25 kB/sec 2024-09-19T07:35:35.314 INFO:tasks.workunit.client.0.smithi159.stdout: Min xfer = 1048576.00 kB 2024-09-19T07:35:35.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:35 smithi159 ceph-mon[30472]: pgmap v452: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 155 MiB/s wr, 44 op/s 2024-09-19T07:35:38.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:37 smithi159 ceph-mon[30472]: pgmap v453: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 125 MiB/s wr, 39 op/s 2024-09-19T07:35:38.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:37 smithi203 ceph-mon[35335]: pgmap v453: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 125 MiB/s wr, 39 op/s 2024-09-19T07:35:38.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:37 smithi189 ceph-mon[34851]: pgmap v453: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 125 MiB/s wr, 39 op/s 2024-09-19T07:35:40.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:39 smithi159 ceph-mon[30472]: pgmap v454: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 92 MiB/s wr, 28 op/s 2024-09-19T07:35:40.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:39 smithi203 ceph-mon[35335]: pgmap v454: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 92 MiB/s wr, 28 op/s 2024-09-19T07:35:40.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:39 smithi189 ceph-mon[34851]: pgmap v454: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 92 MiB/s wr, 28 op/s 2024-09-19T07:35:41.776 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:41.776 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:41.776 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:41.777 INFO:tasks.workunit.client.0.smithi159.stdout:iozone test complete. 2024-09-19T07:35:41.779 INFO:tasks.workunit.client.0.smithi159.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-19T07:35:41.858 INFO:tasks.workunit.client.0.smithi159.stdout: Iozone: Performance Test of File I/O 2024-09-19T07:35:41.858 INFO:tasks.workunit.client.0.smithi159.stdout: Version $Revision: 3.494 $ 2024-09-19T07:35:41.858 INFO:tasks.workunit.client.0.smithi159.stdout: Compiled for 64 bit mode. 2024-09-19T07:35:41.858 INFO:tasks.workunit.client.0.smithi159.stdout: Build: linux-AMD64 2024-09-19T07:35:41.858 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:41.859 INFO:tasks.workunit.client.0.smithi159.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-19T07:35:41.859 INFO:tasks.workunit.client.0.smithi159.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-19T07:35:41.859 INFO:tasks.workunit.client.0.smithi159.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-19T07:35:41.859 INFO:tasks.workunit.client.0.smithi159.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-19T07:35:41.859 INFO:tasks.workunit.client.0.smithi159.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-19T07:35:41.859 INFO:tasks.workunit.client.0.smithi159.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-19T07:35:41.859 INFO:tasks.workunit.client.0.smithi159.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-19T07:35:41.859 INFO:tasks.workunit.client.0.smithi159.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-19T07:35:41.859 INFO:tasks.workunit.client.0.smithi159.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-19T07:35:41.859 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:41.859 INFO:tasks.workunit.client.0.smithi159.stdout: Run began: Thu Sep 19 07:35:41 2024 2024-09-19T07:35:41.859 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:35:41.859 INFO:tasks.workunit.client.0.smithi159.stdout: Include close in write timing 2024-09-19T07:35:41.859 INFO:tasks.workunit.client.0.smithi159.stdout: Include fsync in write timing 2024-09-19T07:35:41.860 INFO:tasks.workunit.client.0.smithi159.stdout: File size set to 10485760 kB 2024-09-19T07:35:41.860 INFO:tasks.workunit.client.0.smithi159.stdout: Record Size 1024 kB 2024-09-19T07:35:41.860 INFO:tasks.workunit.client.0.smithi159.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-19T07:35:41.860 INFO:tasks.workunit.client.0.smithi159.stdout: Output is in kBytes/sec 2024-09-19T07:35:41.860 INFO:tasks.workunit.client.0.smithi159.stdout: Time Resolution = 0.000001 seconds. 2024-09-19T07:35:41.860 INFO:tasks.workunit.client.0.smithi159.stdout: Processor cache size set to 1024 kBytes. 2024-09-19T07:35:41.860 INFO:tasks.workunit.client.0.smithi159.stdout: Processor cache line size set to 32 bytes. 2024-09-19T07:35:41.860 INFO:tasks.workunit.client.0.smithi159.stdout: File stride size set to 17 * record size. 2024-09-19T07:35:41.860 INFO:tasks.workunit.client.0.smithi159.stdout: Throughput test with 1 process 2024-09-19T07:35:41.860 INFO:tasks.workunit.client.0.smithi159.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-09-19T07:35:42.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:41 smithi159 ceph-mon[30472]: pgmap v455: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 75 MiB/s wr, 24 op/s 2024-09-19T07:35:42.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:41 smithi203 ceph-mon[35335]: pgmap v455: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 75 MiB/s wr, 24 op/s 2024-09-19T07:35:42.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:41 smithi189 ceph-mon[34851]: pgmap v455: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 75 MiB/s wr, 24 op/s 2024-09-19T07:35:44.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:43 smithi159 ceph-mon[30472]: pgmap v456: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 54 MiB/s wr, 21 op/s 2024-09-19T07:35:44.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:43 smithi203 ceph-mon[35335]: pgmap v456: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 54 MiB/s wr, 21 op/s 2024-09-19T07:35:44.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:43 smithi189 ceph-mon[34851]: pgmap v456: 129 pgs: 129 active+clean; 1.0 GiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 54 MiB/s wr, 21 op/s 2024-09-19T07:35:46.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:45 smithi159 ceph-mon[30472]: pgmap v457: 129 pgs: 129 active+clean; 633 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 11 MiB/s wr, 16 op/s 2024-09-19T07:35:46.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:45 smithi203 ceph-mon[35335]: pgmap v457: 129 pgs: 129 active+clean; 633 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 11 MiB/s wr, 16 op/s 2024-09-19T07:35:46.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:45 smithi189 ceph-mon[34851]: pgmap v457: 129 pgs: 129 active+clean; 633 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 11 MiB/s wr, 16 op/s 2024-09-19T07:35:48.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:47 smithi159 ceph-mon[30472]: pgmap v458: 129 pgs: 129 active+clean; 297 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 30 op/s 2024-09-19T07:35:48.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:47 smithi203 ceph-mon[35335]: pgmap v458: 129 pgs: 129 active+clean; 297 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 30 op/s 2024-09-19T07:35:48.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:47 smithi189 ceph-mon[34851]: pgmap v458: 129 pgs: 129 active+clean; 297 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 30 op/s 2024-09-19T07:35:50.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:49 smithi203 ceph-mon[35335]: pgmap v459: 129 pgs: 129 active+clean; 369 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 31 MiB/s wr, 37 op/s 2024-09-19T07:35:50.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:49 smithi189 ceph-mon[34851]: pgmap v459: 129 pgs: 129 active+clean; 369 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 31 MiB/s wr, 37 op/s 2024-09-19T07:35:50.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:49 smithi159 ceph-mon[30472]: pgmap v459: 129 pgs: 129 active+clean; 369 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 31 MiB/s wr, 37 op/s 2024-09-19T07:35:52.190 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:51 smithi159 ceph-mon[30472]: pgmap v460: 129 pgs: 129 active+clean; 897 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 75 MiB/s wr, 48 op/s 2024-09-19T07:35:52.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:51 smithi203 ceph-mon[35335]: pgmap v460: 129 pgs: 129 active+clean; 897 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 75 MiB/s wr, 48 op/s 2024-09-19T07:35:52.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:51 smithi189 ceph-mon[34851]: pgmap v460: 129 pgs: 129 active+clean; 897 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 75 MiB/s wr, 48 op/s 2024-09-19T07:35:54.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:53 smithi203 ceph-mon[35335]: pgmap v461: 129 pgs: 129 active+clean; 1.7 GiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 149 MiB/s wr, 70 op/s 2024-09-19T07:35:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:53 smithi189 ceph-mon[34851]: pgmap v461: 129 pgs: 129 active+clean; 1.7 GiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 149 MiB/s wr, 70 op/s 2024-09-19T07:35:54.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:53 smithi159 ceph-mon[30472]: pgmap v461: 129 pgs: 129 active+clean; 1.7 GiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 149 MiB/s wr, 70 op/s 2024-09-19T07:35:56.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:55 smithi203 ceph-mon[35335]: pgmap v462: 129 pgs: 129 active+clean; 2.2 GiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 188 MiB/s wr, 77 op/s 2024-09-19T07:35:56.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:55 smithi189 ceph-mon[34851]: pgmap v462: 129 pgs: 129 active+clean; 2.2 GiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 188 MiB/s wr, 77 op/s 2024-09-19T07:35:56.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:55 smithi159 ceph-mon[30472]: pgmap v462: 129 pgs: 129 active+clean; 2.2 GiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 188 MiB/s wr, 77 op/s 2024-09-19T07:35:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:57 smithi203 ceph-mon[35335]: pgmap v463: 129 pgs: 129 active+clean; 2.9 GiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 249 MiB/s wr, 88 op/s 2024-09-19T07:35:58.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:57 smithi189 ceph-mon[34851]: pgmap v463: 129 pgs: 129 active+clean; 2.9 GiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 249 MiB/s wr, 88 op/s 2024-09-19T07:35:58.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:57 smithi159 ceph-mon[30472]: pgmap v463: 129 pgs: 129 active+clean; 2.9 GiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 249 MiB/s wr, 88 op/s 2024-09-19T07:36:00.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:35:59 smithi203 ceph-mon[35335]: pgmap v464: 129 pgs: 129 active+clean; 3.3 GiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 275 MiB/s wr, 80 op/s 2024-09-19T07:36:00.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:35:59 smithi189 ceph-mon[34851]: pgmap v464: 129 pgs: 129 active+clean; 3.3 GiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 275 MiB/s wr, 80 op/s 2024-09-19T07:36:00.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:35:59 smithi159 ceph-mon[30472]: pgmap v464: 129 pgs: 129 active+clean; 3.3 GiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 275 MiB/s wr, 80 op/s 2024-09-19T07:36:02.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:01 smithi203 ceph-mon[35335]: pgmap v465: 129 pgs: 129 active+clean; 3.8 GiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 295 MiB/s wr, 81 op/s 2024-09-19T07:36:02.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:01 smithi189 ceph-mon[34851]: pgmap v465: 129 pgs: 129 active+clean; 3.8 GiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 295 MiB/s wr, 81 op/s 2024-09-19T07:36:02.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:01 smithi159 ceph-mon[30472]: pgmap v465: 129 pgs: 129 active+clean; 3.8 GiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 295 MiB/s wr, 81 op/s 2024-09-19T07:36:04.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:03 smithi203 ceph-mon[35335]: pgmap v466: 129 pgs: 129 active+clean; 4.6 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 318 MiB/s wr, 91 op/s 2024-09-19T07:36:04.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:03 smithi189 ceph-mon[34851]: pgmap v466: 129 pgs: 129 active+clean; 4.6 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 318 MiB/s wr, 91 op/s 2024-09-19T07:36:04.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:03 smithi159 ceph-mon[30472]: pgmap v466: 129 pgs: 129 active+clean; 4.6 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 318 MiB/s wr, 91 op/s 2024-09-19T07:36:06.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:05 smithi203 ceph-mon[35335]: pgmap v467: 129 pgs: 129 active+clean; 5.0 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 276 MiB/s wr, 76 op/s 2024-09-19T07:36:06.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:05 smithi189 ceph-mon[34851]: pgmap v467: 129 pgs: 129 active+clean; 5.0 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 276 MiB/s wr, 76 op/s 2024-09-19T07:36:06.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:05 smithi159 ceph-mon[30472]: pgmap v467: 129 pgs: 129 active+clean; 5.0 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 276 MiB/s wr, 76 op/s 2024-09-19T07:36:08.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:07 smithi203 ceph-mon[35335]: pgmap v468: 129 pgs: 129 active+clean; 5.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 298 MiB/s wr, 84 op/s 2024-09-19T07:36:08.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:07 smithi189 ceph-mon[34851]: pgmap v468: 129 pgs: 129 active+clean; 5.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 298 MiB/s wr, 84 op/s 2024-09-19T07:36:08.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:07 smithi159 ceph-mon[30472]: pgmap v468: 129 pgs: 129 active+clean; 5.7 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 298 MiB/s wr, 84 op/s 2024-09-19T07:36:10.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:09 smithi203 ceph-mon[35335]: pgmap v469: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 279 MiB/s wr, 77 op/s 2024-09-19T07:36:10.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:09 smithi189 ceph-mon[34851]: pgmap v469: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 279 MiB/s wr, 77 op/s 2024-09-19T07:36:10.394 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:09 smithi159 ceph-mon[30472]: pgmap v469: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 279 MiB/s wr, 77 op/s 2024-09-19T07:36:11.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:10 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:36:11.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:10 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:36:11.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:10 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:36:12.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:11 smithi203 ceph-mon[35335]: pgmap v470: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 290 MiB/s wr, 79 op/s 2024-09-19T07:36:12.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:11 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:36:12.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:11 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:36:12.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:11 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:36:12.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:11 smithi189 ceph-mon[34851]: pgmap v470: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 290 MiB/s wr, 79 op/s 2024-09-19T07:36:12.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:11 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:36:12.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:11 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:36:12.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:11 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:36:12.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:11 smithi159 ceph-mon[30472]: pgmap v470: 129 pgs: 129 active+clean; 6.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 290 MiB/s wr, 79 op/s 2024-09-19T07:36:12.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:11 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:36:12.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:11 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:36:12.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:11 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:36:14.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:13 smithi203 ceph-mon[35335]: pgmap v471: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 326 MiB/s wr, 91 op/s 2024-09-19T07:36:14.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:13 smithi189 ceph-mon[34851]: pgmap v471: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 326 MiB/s wr, 91 op/s 2024-09-19T07:36:14.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:13 smithi159 ceph-mon[30472]: pgmap v471: 129 pgs: 129 active+clean; 7.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 326 MiB/s wr, 91 op/s 2024-09-19T07:36:16.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:15 smithi203 ceph-mon[35335]: pgmap v472: 129 pgs: 129 active+clean; 8.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 297 MiB/s wr, 80 op/s 2024-09-19T07:36:16.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:15 smithi189 ceph-mon[34851]: pgmap v472: 129 pgs: 129 active+clean; 8.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 297 MiB/s wr, 80 op/s 2024-09-19T07:36:16.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:15 smithi159 ceph-mon[30472]: pgmap v472: 129 pgs: 129 active+clean; 8.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 297 MiB/s wr, 80 op/s 2024-09-19T07:36:17.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:17 smithi203 ceph-mon[35335]: pgmap v473: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 322 MiB/s wr, 88 op/s 2024-09-19T07:36:17.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:17 smithi189 ceph-mon[34851]: pgmap v473: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 322 MiB/s wr, 88 op/s 2024-09-19T07:36:17.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:17 smithi159 ceph-mon[30472]: pgmap v473: 129 pgs: 129 active+clean; 8.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 322 MiB/s wr, 88 op/s 2024-09-19T07:36:18.441 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:36:18.441 INFO:tasks.workunit.client.0.smithi159.stdout: Children see throughput for 1 initial writers = 286897.44 kB/sec 2024-09-19T07:36:18.441 INFO:tasks.workunit.client.0.smithi159.stdout: Parent sees throughput for 1 initial writers = 286889.48 kB/sec 2024-09-19T07:36:18.441 INFO:tasks.workunit.client.0.smithi159.stdout: Min throughput per process = 286897.44 kB/sec 2024-09-19T07:36:18.441 INFO:tasks.workunit.client.0.smithi159.stdout: Max throughput per process = 286897.44 kB/sec 2024-09-19T07:36:18.441 INFO:tasks.workunit.client.0.smithi159.stdout: Avg throughput per process = 286897.44 kB/sec 2024-09-19T07:36:18.442 INFO:tasks.workunit.client.0.smithi159.stdout: Min xfer = 10485760.00 kB 2024-09-19T07:36:20.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:19 smithi159 ceph-mon[30472]: pgmap v474: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 300 MiB/s wr, 80 op/s 2024-09-19T07:36:20.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:19 smithi203 ceph-mon[35335]: pgmap v474: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 300 MiB/s wr, 80 op/s 2024-09-19T07:36:20.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:19 smithi189 ceph-mon[34851]: pgmap v474: 129 pgs: 129 active+clean; 9.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 300 MiB/s wr, 80 op/s 2024-09-19T07:36:22.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:21 smithi159 ceph-mon[30472]: pgmap v475: 129 pgs: 129 active+clean; 9.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 299 MiB/s wr, 80 op/s 2024-09-19T07:36:22.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:21 smithi203 ceph-mon[35335]: pgmap v475: 129 pgs: 129 active+clean; 9.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 299 MiB/s wr, 80 op/s 2024-09-19T07:36:22.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:21 smithi189 ceph-mon[34851]: pgmap v475: 129 pgs: 129 active+clean; 9.7 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 299 MiB/s wr, 80 op/s 2024-09-19T07:36:24.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:23 smithi159 ceph-mon[30472]: pgmap v476: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 277 MiB/s wr, 77 op/s 2024-09-19T07:36:24.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:23 smithi203 ceph-mon[35335]: pgmap v476: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 277 MiB/s wr, 77 op/s 2024-09-19T07:36:24.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:23 smithi189 ceph-mon[34851]: pgmap v476: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 277 MiB/s wr, 77 op/s 2024-09-19T07:36:26.108 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:25 smithi189 ceph-mon[34851]: pgmap v477: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 215 MiB/s wr, 59 op/s 2024-09-19T07:36:26.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:25 smithi159 ceph-mon[30472]: pgmap v477: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 215 MiB/s wr, 59 op/s 2024-09-19T07:36:26.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:25 smithi203 ceph-mon[35335]: pgmap v477: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 215 MiB/s wr, 59 op/s 2024-09-19T07:36:28.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:27 smithi159 ceph-mon[30472]: pgmap v478: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 220 MiB/s wr, 63 op/s 2024-09-19T07:36:28.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:27 smithi203 ceph-mon[35335]: pgmap v478: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 220 MiB/s wr, 63 op/s 2024-09-19T07:36:28.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:27 smithi189 ceph-mon[34851]: pgmap v478: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 220 MiB/s wr, 63 op/s 2024-09-19T07:36:30.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:29 smithi159 ceph-mon[30472]: pgmap v479: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 199 MiB/s wr, 56 op/s 2024-09-19T07:36:30.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:29 smithi203 ceph-mon[35335]: pgmap v479: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 199 MiB/s wr, 56 op/s 2024-09-19T07:36:30.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:29 smithi189 ceph-mon[34851]: pgmap v479: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 199 MiB/s wr, 56 op/s 2024-09-19T07:36:32.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:31 smithi159 ceph-mon[30472]: pgmap v480: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 218 MiB/s wr, 61 op/s 2024-09-19T07:36:32.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:31 smithi203 ceph-mon[35335]: pgmap v480: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 218 MiB/s wr, 61 op/s 2024-09-19T07:36:32.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:31 smithi189 ceph-mon[34851]: pgmap v480: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 218 MiB/s wr, 61 op/s 2024-09-19T07:36:34.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:33 smithi159 ceph-mon[30472]: pgmap v481: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 245 MiB/s wr, 71 op/s 2024-09-19T07:36:34.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:33 smithi203 ceph-mon[35335]: pgmap v481: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 245 MiB/s wr, 71 op/s 2024-09-19T07:36:34.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:33 smithi189 ceph-mon[34851]: pgmap v481: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 245 MiB/s wr, 71 op/s 2024-09-19T07:36:36.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:35 smithi159 ceph-mon[30472]: pgmap v482: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 257 MiB/s wr, 71 op/s 2024-09-19T07:36:36.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:35 smithi203 ceph-mon[35335]: pgmap v482: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 257 MiB/s wr, 71 op/s 2024-09-19T07:36:36.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:35 smithi189 ceph-mon[34851]: pgmap v482: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 257 MiB/s wr, 71 op/s 2024-09-19T07:36:38.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:37 smithi159 ceph-mon[30472]: pgmap v483: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 301 MiB/s wr, 86 op/s 2024-09-19T07:36:38.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:37 smithi203 ceph-mon[35335]: pgmap v483: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 301 MiB/s wr, 86 op/s 2024-09-19T07:36:38.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:37 smithi189 ceph-mon[34851]: pgmap v483: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 301 MiB/s wr, 86 op/s 2024-09-19T07:36:40.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:39 smithi159 ceph-mon[30472]: pgmap v484: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 293 MiB/s wr, 81 op/s 2024-09-19T07:36:40.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:39 smithi203 ceph-mon[35335]: pgmap v484: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 293 MiB/s wr, 81 op/s 2024-09-19T07:36:40.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:39 smithi189 ceph-mon[34851]: pgmap v484: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 293 MiB/s wr, 81 op/s 2024-09-19T07:36:42.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:41 smithi203 ceph-mon[35335]: pgmap v485: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 316 MiB/s wr, 86 op/s 2024-09-19T07:36:42.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:41 smithi189 ceph-mon[34851]: pgmap v485: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 316 MiB/s wr, 86 op/s 2024-09-19T07:36:42.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:41 smithi159 ceph-mon[30472]: pgmap v485: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 316 MiB/s wr, 86 op/s 2024-09-19T07:36:44.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:43 smithi203 ceph-mon[35335]: pgmap v486: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 321 MiB/s wr, 91 op/s 2024-09-19T07:36:44.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:43 smithi189 ceph-mon[34851]: pgmap v486: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 321 MiB/s wr, 91 op/s 2024-09-19T07:36:44.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:43 smithi159 ceph-mon[30472]: pgmap v486: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 321 MiB/s wr, 91 op/s 2024-09-19T07:36:46.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:45 smithi203 ceph-mon[35335]: pgmap v487: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 299 MiB/s wr, 82 op/s 2024-09-19T07:36:46.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:45 smithi189 ceph-mon[34851]: pgmap v487: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 299 MiB/s wr, 82 op/s 2024-09-19T07:36:46.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:45 smithi159 ceph-mon[30472]: pgmap v487: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 299 MiB/s wr, 82 op/s 2024-09-19T07:36:48.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:47 smithi203 ceph-mon[35335]: pgmap v488: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 328 MiB/s wr, 91 op/s 2024-09-19T07:36:48.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:47 smithi189 ceph-mon[34851]: pgmap v488: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 328 MiB/s wr, 91 op/s 2024-09-19T07:36:48.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:47 smithi159 ceph-mon[30472]: pgmap v488: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 328 MiB/s wr, 91 op/s 2024-09-19T07:36:50.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:49 smithi203 ceph-mon[35335]: pgmap v489: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 308 MiB/s wr, 83 op/s 2024-09-19T07:36:50.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:49 smithi189 ceph-mon[34851]: pgmap v489: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 308 MiB/s wr, 83 op/s 2024-09-19T07:36:50.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:49 smithi159 ceph-mon[30472]: pgmap v489: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 308 MiB/s wr, 83 op/s 2024-09-19T07:36:52.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:51 smithi203 ceph-mon[35335]: pgmap v490: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 332 MiB/s wr, 89 op/s 2024-09-19T07:36:52.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:51 smithi189 ceph-mon[34851]: pgmap v490: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 332 MiB/s wr, 89 op/s 2024-09-19T07:36:52.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:51 smithi159 ceph-mon[30472]: pgmap v490: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 332 MiB/s wr, 89 op/s 2024-09-19T07:36:54.189 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:53 smithi159 ceph-mon[30472]: pgmap v491: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 339 MiB/s wr, 93 op/s 2024-09-19T07:36:54.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:53 smithi203 ceph-mon[35335]: pgmap v491: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 339 MiB/s wr, 93 op/s 2024-09-19T07:36:54.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:53 smithi189 ceph-mon[34851]: pgmap v491: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 339 MiB/s wr, 93 op/s 2024-09-19T07:36:54.665 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:36:54.665 INFO:tasks.workunit.client.0.smithi159.stdout: Children see throughput for 1 rewriters = 311496.00 kB/sec 2024-09-19T07:36:54.665 INFO:tasks.workunit.client.0.smithi159.stdout: Parent sees throughput for 1 rewriters = 311489.85 kB/sec 2024-09-19T07:36:54.665 INFO:tasks.workunit.client.0.smithi159.stdout: Min throughput per process = 311496.00 kB/sec 2024-09-19T07:36:54.665 INFO:tasks.workunit.client.0.smithi159.stdout: Max throughput per process = 311496.00 kB/sec 2024-09-19T07:36:54.665 INFO:tasks.workunit.client.0.smithi159.stdout: Avg throughput per process = 311496.00 kB/sec 2024-09-19T07:36:54.665 INFO:tasks.workunit.client.0.smithi159.stdout: Min xfer = 10485760.00 kB 2024-09-19T07:36:56.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:55 smithi203 ceph-mon[35335]: pgmap v492: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 315 MiB/s wr, 84 op/s 2024-09-19T07:36:56.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:55 smithi189 ceph-mon[34851]: pgmap v492: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 315 MiB/s wr, 84 op/s 2024-09-19T07:36:56.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:55 smithi159 ceph-mon[30472]: pgmap v492: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 315 MiB/s wr, 84 op/s 2024-09-19T07:36:58.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:57 smithi203 ceph-mon[35335]: pgmap v493: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 313 MiB/s wr, 85 op/s 2024-09-19T07:36:58.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:57 smithi189 ceph-mon[34851]: pgmap v493: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 313 MiB/s wr, 85 op/s 2024-09-19T07:36:58.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:57 smithi159 ceph-mon[30472]: pgmap v493: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 313 MiB/s wr, 85 op/s 2024-09-19T07:37:00.205 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:37:00.205 INFO:tasks.workunit.client.0.smithi159.stdout: Children see throughput for 1 readers = 5090367.50 kB/sec 2024-09-19T07:37:00.205 INFO:tasks.workunit.client.0.smithi159.stdout: Parent sees throughput for 1 readers = 5088445.94 kB/sec 2024-09-19T07:37:00.205 INFO:tasks.workunit.client.0.smithi159.stdout: Min throughput per process = 5090367.50 kB/sec 2024-09-19T07:37:00.205 INFO:tasks.workunit.client.0.smithi159.stdout: Max throughput per process = 5090367.50 kB/sec 2024-09-19T07:37:00.205 INFO:tasks.workunit.client.0.smithi159.stdout: Avg throughput per process = 5090367.50 kB/sec 2024-09-19T07:37:00.205 INFO:tasks.workunit.client.0.smithi159.stdout: Min xfer = 10485760.00 kB 2024-09-19T07:37:00.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:36:59 smithi203 ceph-mon[35335]: pgmap v494: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 264 MiB/s wr, 71 op/s 2024-09-19T07:37:00.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:36:59 smithi189 ceph-mon[34851]: pgmap v494: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 264 MiB/s wr, 71 op/s 2024-09-19T07:37:00.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:36:59 smithi159 ceph-mon[30472]: pgmap v494: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 264 MiB/s wr, 71 op/s 2024-09-19T07:37:02.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:01 smithi203 ceph-mon[35335]: pgmap v495: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 232 MiB/s wr, 63 op/s 2024-09-19T07:37:02.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:01 smithi189 ceph-mon[34851]: pgmap v495: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 232 MiB/s wr, 63 op/s 2024-09-19T07:37:02.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:01 smithi159 ceph-mon[30472]: pgmap v495: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 232 MiB/s wr, 63 op/s 2024-09-19T07:37:04.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:03 smithi203 ceph-mon[35335]: pgmap v496: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 173 MiB/s wr, 50 op/s 2024-09-19T07:37:04.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:03 smithi189 ceph-mon[34851]: pgmap v496: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 173 MiB/s wr, 50 op/s 2024-09-19T07:37:04.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:03 smithi159 ceph-mon[30472]: pgmap v496: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 173 MiB/s wr, 50 op/s 2024-09-19T07:37:05.275 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:37:05.275 INFO:tasks.workunit.client.0.smithi159.stdout: Children see throughput for 1 re-readers = 3527061.50 kB/sec 2024-09-19T07:37:05.275 INFO:tasks.workunit.client.0.smithi159.stdout: Parent sees throughput for 1 re-readers = 3526228.82 kB/sec 2024-09-19T07:37:05.275 INFO:tasks.workunit.client.0.smithi159.stdout: Min throughput per process = 3527061.50 kB/sec 2024-09-19T07:37:05.275 INFO:tasks.workunit.client.0.smithi159.stdout: Max throughput per process = 3527061.50 kB/sec 2024-09-19T07:37:05.275 INFO:tasks.workunit.client.0.smithi159.stdout: Avg throughput per process = 3527061.50 kB/sec 2024-09-19T07:37:05.275 INFO:tasks.workunit.client.0.smithi159.stdout: Min xfer = 10485760.00 kB 2024-09-19T07:37:06.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:05 smithi203 ceph-mon[35335]: pgmap v497: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 106 MiB/s wr, 31 op/s 2024-09-19T07:37:06.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:05 smithi189 ceph-mon[34851]: pgmap v497: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 106 MiB/s wr, 31 op/s 2024-09-19T07:37:06.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:05 smithi159 ceph-mon[30472]: pgmap v497: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 106 MiB/s wr, 31 op/s 2024-09-19T07:37:08.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:07 smithi203 ceph-mon[35335]: pgmap v498: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 66 MiB/s wr, 23 op/s 2024-09-19T07:37:08.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:07 smithi189 ceph-mon[34851]: pgmap v498: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 66 MiB/s wr, 23 op/s 2024-09-19T07:37:08.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:07 smithi159 ceph-mon[30472]: pgmap v498: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 66 MiB/s wr, 23 op/s 2024-09-19T07:37:10.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:09 smithi203 ceph-mon[35335]: pgmap v499: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 22 MiB/s wr, 11 op/s 2024-09-19T07:37:10.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:09 smithi189 ceph-mon[34851]: pgmap v499: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 22 MiB/s wr, 11 op/s 2024-09-19T07:37:10.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:09 smithi159 ceph-mon[30472]: pgmap v499: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 22 MiB/s wr, 11 op/s 2024-09-19T07:37:12.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:11 smithi203 ceph-mon[35335]: pgmap v500: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4.7 MiB/s wr, 6 op/s 2024-09-19T07:37:12.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:11 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:37:12.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:11 smithi189 ceph-mon[34851]: pgmap v500: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4.7 MiB/s wr, 6 op/s 2024-09-19T07:37:12.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:11 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:37:12.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:11 smithi159 ceph-mon[30472]: pgmap v500: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4.7 MiB/s wr, 6 op/s 2024-09-19T07:37:12.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:11 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:37:13.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:37:13.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:37:13.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:12 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:37:13.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:37:13.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:37:13.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:12 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:37:13.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:37:13.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:37:13.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:12 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' 2024-09-19T07:37:13.589 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:37:13.589 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:37:13.590 INFO:tasks.workunit.client.0.smithi159.stdout: 2024-09-19T07:37:13.590 INFO:tasks.workunit.client.0.smithi159.stdout:iozone test complete. 2024-09-19T07:37:13.594 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T07:37:13.594 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T07:37:13.710 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-09-19T07:37:13.710 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-19T07:37:14.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:13 smithi203 ceph-mon[35335]: pgmap v501: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-19T07:37:14.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:13 smithi189 ceph-mon[34851]: pgmap v501: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-19T07:37:14.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:13 smithi159 ceph-mon[30472]: pgmap v501: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-19T07:37:15.713 DEBUG:teuthology.parallel:result is None 2024-09-19T07:37:15.713 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T07:37:15.789 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T07:37:15.789 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-19T07:37:15.800 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:37:15.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.956+0000 7f91b9600640 1 -- 172.21.15.159:0/270100011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91b4103d30 msgr2=0x7f91b4104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:15.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.956+0000 7f91b9600640 1 --2- 172.21.15.159:0/270100011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91b4103d30 0x7f91b4104130 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009900 tx=0x7f91a402a3f0 comp rx=0 tx=0).stop 2024-09-19T07:37:15.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.957+0000 7f91b9600640 1 -- 172.21.15.159:0/270100011 shutdown_connections 2024-09-19T07:37:15.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.957+0000 7f91b9600640 1 --2- 172.21.15.159:0/270100011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91b41058e0 0x7f91b410c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:15.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.957+0000 7f91b9600640 1 --2- 172.21.15.159:0/270100011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91b4104f20 0x7f91b41053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:15.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.957+0000 7f91b9600640 1 --2- 172.21.15.159:0/270100011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91b4103d30 0x7f91b4104130 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:15.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.957+0000 7f91b9600640 1 -- 172.21.15.159:0/270100011 >> 172.21.15.159:0/270100011 conn(0x7f91b40ff4b0 msgr2=0x7f91b41018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:15.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.957+0000 7f91b9600640 1 -- 172.21.15.159:0/270100011 shutdown_connections 2024-09-19T07:37:15.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.957+0000 7f91b9600640 1 -- 172.21.15.159:0/270100011 wait complete. 2024-09-19T07:37:15.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.958+0000 7f91b9600640 1 Processor -- start 2024-09-19T07:37:15.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.958+0000 7f91b9600640 1 -- start start 2024-09-19T07:37:15.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.958+0000 7f91b9600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91b4103d30 0x7f91b419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:15.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.958+0000 7f91b9600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91b4104f20 0x7f91b419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:15.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.958+0000 7f91b9600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91b41058e0 0x7f91b41a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:15.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.958+0000 7f91b9600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f91b410fa00 con 0x7f91b41058e0 2024-09-19T07:37:15.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.958+0000 7f91b9600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f91b410f880 con 0x7f91b4103d30 2024-09-19T07:37:15.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.958+0000 7f91b9600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f91b410fb80 con 0x7f91b4104f20 2024-09-19T07:37:15.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.958+0000 7f91b2a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91b4103d30 0x7f91b419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:15.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.958+0000 7f91b3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91b41058e0 0x7f91b41a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:15.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.958+0000 7f91b2a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91b4103d30 0x7f91b419a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:46484/0 (socket says 172.21.15.159:46484) 2024-09-19T07:37:15.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.958+0000 7f91b2a00640 1 -- 172.21.15.159:0/602334380 learned_addr learned my addr 172.21.15.159:0/602334380 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:37:15.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.958+0000 7f91aa000640 1 --2- 172.21.15.159:0/602334380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91b4104f20 0x7f91b419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:15.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.959+0000 7f91b2a00640 1 -- 172.21.15.159:0/602334380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91b4104f20 msgr2=0x7f91b419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:15.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.959+0000 7f91b2a00640 1 --2- 172.21.15.159:0/602334380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91b4104f20 0x7f91b419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:15.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.959+0000 7f91b2a00640 1 -- 172.21.15.159:0/602334380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91b41058e0 msgr2=0x7f91b41a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:15.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.959+0000 7f91b2a00640 1 --2- 172.21.15.159:0/602334380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91b41058e0 0x7f91b41a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:15.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.959+0000 7f91b2a00640 1 -- 172.21.15.159:0/602334380 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b41a1c50 con 0x7f91b4103d30 2024-09-19T07:37:15.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.959+0000 7f91b2a00640 1 --2- 172.21.15.159:0/602334380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91b4103d30 0x7f91b419a030 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f91a4031440 tx=0x7f91a4032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:15.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.959+0000 7f91abe00640 1 -- 172.21.15.159:0/602334380 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a403f030 con 0x7f91b4103d30 2024-09-19T07:37:15.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.959+0000 7f91abe00640 1 -- 172.21.15.159:0/602334380 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91a4003ce0 con 0x7f91b4103d30 2024-09-19T07:37:15.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.959+0000 7f91abe00640 1 -- 172.21.15.159:0/602334380 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4035070 con 0x7f91b4103d30 2024-09-19T07:37:15.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.959+0000 7f91b9600640 1 -- 172.21.15.159:0/602334380 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b41a1ee0 con 0x7f91b4103d30 2024-09-19T07:37:15.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.960+0000 7f91b9600640 1 -- 172.21.15.159:0/602334380 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b41a2340 con 0x7f91b4103d30 2024-09-19T07:37:15.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.960+0000 7f91b9600640 1 -- 172.21.15.159:0/602334380 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b410b0d0 con 0x7f91b4103d30 2024-09-19T07:37:15.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.962+0000 7f91abe00640 1 -- 172.21.15.159:0/602334380 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f91a4042030 con 0x7f91b4103d30 2024-09-19T07:37:15.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.963+0000 7f91abe00640 1 --2- 172.21.15.159:0/602334380 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f91980a32d0 0x7f91980a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:15.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.963+0000 7f91abe00640 1 -- 172.21.15.159:0/602334380 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f91a40f00f0 con 0x7f91b4103d30 2024-09-19T07:37:15.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.963+0000 7f91aa000640 1 --2- 172.21.15.159:0/602334380 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f91980a32d0 0x7f91980a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:15.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.964+0000 7f91aa000640 1 --2- 172.21.15.159:0/602334380 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f91980a32d0 0x7f91980a5790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f91b419b540 tx=0x7f919c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:15.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:15.967+0000 7f91abe00640 1 -- 172.21.15.159:0/602334380 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f91a40bc3b0 con 0x7f91b4103d30 2024-09-19T07:37:16.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:15 smithi159 ceph-mon[30472]: pgmap v502: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 17 op/s 2024-09-19T07:37:16.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.146+0000 7f91b9600640 1 -- 172.21.15.159:0/602334380 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91b4062e90 con 0x7f91b4103d30 2024-09-19T07:37:16.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.148+0000 7f91abe00640 1 -- 172.21.15.159:0/602334380 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24963 (secure 0 0 0) 0x7f91a40c06b0 con 0x7f91b4103d30 2024-09-19T07:37:16.149 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:37:16.149 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":100,"fsid":"c9d18048-7657-11ef-bceb-c7b262605968","created":"2024-09-19T07:22:32.059836+0000","modified":"2024-09-19T07:32:31.832496+0000","last_up_change":"2024-09-19T07:30:03.328697+0000","last_in_change":"2024-09-19T07:29:38.987579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:25:55.202457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:30:58.022056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:31:01.042290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:31:10.977289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"752f5466-fe04-408a-8566-86625947fd3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6803","nonce":2738900830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6805","nonce":2738900830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6809","nonce":2738900830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6807","nonce":2738900830}]},"public_addr":"172.21.15.159:6803/2738900830","cluster_addr":"172.21.15.159:6805/2738900830","heartbeat_back_addr":"172.21.15.159:6809/2738900830","heartbeat_front_addr":"172.21.15.159:6807/2738900830","state":["exists","up"]},{"osd":1,"uuid":"7513adda-ca82-4df4-aa99-3b050b410424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6801","nonce":265696178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6803","nonce":265696178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6807","nonce":265696178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6805","nonce":265696178}]},"public_addr":"172.21.15.189:6801/265696178","cluster_addr":"172.21.15.189:6803/265696178","heartbeat_back_addr":"172.21.15.189:6807/265696178","heartbeat_front_addr":"172.21.15.189:6805/265696178","state":["exists","up"]},{"osd":2,"uuid":"aa03be79-58d8-4eb7-a051-0b46c502b12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6801","nonce":1826565288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6803","nonce":1826565288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6807","nonce":1826565288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6805","nonce":1826565288}]},"public_addr":"172.21.15.203:6801/1826565288","cluster_addr":"172.21.15.203:6803/1826565288","heartbeat_back_addr":"172.21.15.203:6807/1826565288","heartbeat_front_addr":"172.21.15.203:6805/1826565288","state":["exists","up"]},{"osd":3,"uuid":"fcd734f1-f40f-4566-82ab-c4607039f23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6811","nonce":109457380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6813","nonce":109457380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6817","nonce":109457380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6815","nonce":109457380}]},"public_addr":"172.21.15.159:6811/109457380","cluster_addr":"172.21.15.159:6813/109457380","heartbeat_back_addr":"172.21.15.159:6817/109457380","heartbeat_front_addr":"172.21.15.159:6815/109457380","state":["exists","up"]},{"osd":4,"uuid":"bc344c9e-2267-4058-bbe1-9c270327db8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6809","nonce":1036888180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6811","nonce":1036888180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6815","nonce":1036888180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6813","nonce":1036888180}]},"public_addr":"172.21.15.189:6809/1036888180","cluster_addr":"172.21.15.189:6811/1036888180","heartbeat_back_addr":"172.21.15.189:6815/1036888180","heartbeat_front_addr":"172.21.15.189:6813/1036888180","state":["exists","up"]},{"osd":5,"uuid":"86c48303-929d-42f4-bf48-cbb32649f95f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6809","nonce":4212601512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6811","nonce":4212601512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6815","nonce":4212601512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6813","nonce":4212601512}]},"public_addr":"172.21.15.203:6809/4212601512","cluster_addr":"172.21.15.203:6811/4212601512","heartbeat_back_addr":"172.21.15.203:6815/4212601512","heartbeat_front_addr":"172.21.15.203:6813/4212601512","state":["exists","up"]},{"osd":6,"uuid":"f0176ed1-c3a7-4d75-b213-fe88fab4441a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6819","nonce":3817684578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6821","nonce":3817684578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6825","nonce":3817684578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6823","nonce":3817684578}]},"public_addr":"172.21.15.159:6819/3817684578","cluster_addr":"172.21.15.159:6821/3817684578","heartbeat_back_addr":"172.21.15.159:6825/3817684578","heartbeat_front_addr":"172.21.15.159:6823/3817684578","state":["exists","up"]},{"osd":7,"uuid":"921bd59d-9415-4559-8e9f-035d336afd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6817","nonce":3936826456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6819","nonce":3936826456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6823","nonce":3936826456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6821","nonce":3936826456}]},"public_addr":"172.21.15.189:6817/3936826456","cluster_addr":"172.21.15.189:6819/3936826456","heartbeat_back_addr":"172.21.15.189:6823/3936826456","heartbeat_front_addr":"172.21.15.189:6821/3936826456","state":["exists","up"]},{"osd":8,"uuid":"ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6817","nonce":4062926645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6819","nonce":4062926645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6823","nonce":4062926645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6821","nonce":4062926645}]},"public_addr":"172.21.15.203:6817/4062926645","cluster_addr":"172.21.15.203:6819/4062926645","heartbeat_back_addr":"172.21.15.203:6823/4062926645","heartbeat_front_addr":"172.21.15.203:6821/4062926645","state":["exists","up"]},{"osd":9,"uuid":"df29a10f-1c60-41cb-9c2d-bc9f1cb174bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6827","nonce":4185848747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6829","nonce":4185848747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6833","nonce":4185848747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6831","nonce":4185848747}]},"public_addr":"172.21.15.159:6827/4185848747","cluster_addr":"172.21.15.159:6829/4185848747","heartbeat_back_addr":"172.21.15.159:6833/4185848747","heartbeat_front_addr":"172.21.15.159:6831/4185848747","state":["exists","up"]},{"osd":10,"uuid":"ccc440b4-4682-4998-bb53-95e24423cd1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6825","nonce":3018377605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6827","nonce":3018377605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6831","nonce":3018377605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6829","nonce":3018377605}]},"public_addr":"172.21.15.189:6825/3018377605","cluster_addr":"172.21.15.189:6827/3018377605","heartbeat_back_addr":"172.21.15.189:6831/3018377605","heartbeat_front_addr":"172.21.15.189:6829/3018377605","state":["exists","up"]},{"osd":11,"uuid":"e84afc5f-8051-4341-8fd0-3afb482ab4df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6825","nonce":2807503891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6827","nonce":2807503891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6831","nonce":2807503891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6829","nonce":2807503891}]},"public_addr":"172.21.15.203:6825/2807503891","cluster_addr":"172.21.15.203:6827/2807503891","heartbeat_back_addr":"172.21.15.203:6831/2807503891","heartbeat_front_addr":"172.21.15.203:6829/2807503891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:00.093677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:25.477205+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:51.345020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:21.077603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:48.822640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:13.516029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:44.382228+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:10.810700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:36.817155+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:07.728553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:34.747540+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:01.249332+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:0/2383222735":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1561371050":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6800/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:6800/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/2108730247":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/3393992615":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1366863724":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/1628344860":"2024-09-20T07:23:03.089190+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:37:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.150+0000 7f91b9600640 1 -- 172.21.15.159:0/602334380 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f91980a32d0 msgr2=0x7f91980a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.150+0000 7f91b9600640 1 --2- 172.21.15.159:0/602334380 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f91980a32d0 0x7f91980a5790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f91b419b540 tx=0x7f919c005f90 comp rx=0 tx=0).stop 2024-09-19T07:37:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.150+0000 7f91b9600640 1 -- 172.21.15.159:0/602334380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91b4103d30 msgr2=0x7f91b419a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.150+0000 7f91b9600640 1 --2- 172.21.15.159:0/602334380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91b4103d30 0x7f91b419a030 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f91a4031440 tx=0x7f91a4032ce0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.150+0000 7f91b9600640 1 -- 172.21.15.159:0/602334380 shutdown_connections 2024-09-19T07:37:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.150+0000 7f91b9600640 1 --2- 172.21.15.159:0/602334380 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f91980a32d0 0x7f91980a5790 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.150+0000 7f91b9600640 1 --2- 172.21.15.159:0/602334380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91b41058e0 0x7f91b41a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.150+0000 7f91b9600640 1 --2- 172.21.15.159:0/602334380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91b4104f20 0x7f91b419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.150+0000 7f91b9600640 1 --2- 172.21.15.159:0/602334380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91b4103d30 0x7f91b419a030 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.150+0000 7f91b9600640 1 -- 172.21.15.159:0/602334380 >> 172.21.15.159:0/602334380 conn(0x7f91b40ff4b0 msgr2=0x7f91b41082a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.150+0000 7f91b9600640 1 -- 172.21.15.159:0/602334380 shutdown_connections 2024-09-19T07:37:16.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.150+0000 7f91b9600640 1 -- 172.21.15.159:0/602334380 wait complete. 2024-09-19T07:37:16.167 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T07:25:55.202457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T07:30:58.022056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T07:31:01.042290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-19T07:31:10.977289+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-19T07:37:16.167 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T07:37:16.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:15 smithi203 ceph-mon[35335]: pgmap v502: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 17 op/s 2024-09-19T07:37:16.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:15 smithi189 ceph-mon[34851]: pgmap v502: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 17 op/s 2024-09-19T07:37:16.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.326+0000 7f96a8e00640 1 -- 172.21.15.159:0/2565907836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a41058d0 msgr2=0x7f96a410c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:16.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.326+0000 7f96a8e00640 1 --2- 172.21.15.159:0/2565907836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a41058d0 0x7f96a410c160 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f969c009ca0 tx=0x7f969c02a8b0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.327+0000 7f96a8e00640 1 -- 172.21.15.159:0/2565907836 shutdown_connections 2024-09-19T07:37:16.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.327+0000 7f96a8e00640 1 --2- 172.21.15.159:0/2565907836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a41058d0 0x7f96a410c160 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.327+0000 7f96a8e00640 1 --2- 172.21.15.159:0/2565907836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96a4104f10 0x7f96a4105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.327+0000 7f96a8e00640 1 --2- 172.21.15.159:0/2565907836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96a4103d20 0x7f96a4104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.327+0000 7f96a8e00640 1 -- 172.21.15.159:0/2565907836 >> 172.21.15.159:0/2565907836 conn(0x7f96a40ff4a0 msgr2=0x7f96a41018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:16.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.327+0000 7f96a8e00640 1 -- 172.21.15.159:0/2565907836 shutdown_connections 2024-09-19T07:37:16.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.327+0000 7f96a8e00640 1 -- 172.21.15.159:0/2565907836 wait complete. 2024-09-19T07:37:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.328+0000 7f96a8e00640 1 Processor -- start 2024-09-19T07:37:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.328+0000 7f96a8e00640 1 -- start start 2024-09-19T07:37:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.328+0000 7f96a8e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96a4103d20 0x7f96a4199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.328+0000 7f96a8e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96a4104f10 0x7f96a419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.328+0000 7f96a8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a41058d0 0x7f96a41a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.328+0000 7f96a8e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f96a410f9f0 con 0x7f96a41058d0 2024-09-19T07:37:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.328+0000 7f96a8e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96a410f870 con 0x7f96a4104f10 2024-09-19T07:37:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.328+0000 7f96a8e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f96a410fb70 con 0x7f96a4103d20 2024-09-19T07:37:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.328+0000 7f96a1e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96a4103d20 0x7f96a4199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.328+0000 7f96a1e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96a4103d20 0x7f96a4199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:48416/0 (socket says 172.21.15.159:48416) 2024-09-19T07:37:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.328+0000 7f96a1e00640 1 -- 172.21.15.159:0/3885787156 learned_addr learned my addr 172.21.15.159:0/3885787156 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:37:16.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.328+0000 7f96a1400640 1 --2- 172.21.15.159:0/3885787156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96a4104f10 0x7f96a419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:16.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.328+0000 7f96a2800640 1 --2- 172.21.15.159:0/3885787156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a41058d0 0x7f96a41a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:16.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.329+0000 7f96a1e00640 1 -- 172.21.15.159:0/3885787156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96a4104f10 msgr2=0x7f96a419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:16.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.329+0000 7f96a1e00640 1 --2- 172.21.15.159:0/3885787156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96a4104f10 0x7f96a419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.329+0000 7f96a1e00640 1 -- 172.21.15.159:0/3885787156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a41058d0 msgr2=0x7f96a41a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:16.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.329+0000 7f96a1e00640 1 --2- 172.21.15.159:0/3885787156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a41058d0 0x7f96a41a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.329+0000 7f96a1e00640 1 -- 172.21.15.159:0/3885787156 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a41a1c00 con 0x7f96a4103d20 2024-09-19T07:37:16.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.329+0000 7f96a1400640 1 --2- 172.21.15.159:0/3885787156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96a4104f10 0x7f96a419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:37:16.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.329+0000 7f96a1e00640 1 --2- 172.21.15.159:0/3885787156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96a4103d20 0x7f96a4199fe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f969400e9c0 tx=0x7f969400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:16.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.329+0000 7f969aa00640 1 -- 172.21.15.159:0/3885787156 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969400cd80 con 0x7f96a4103d20 2024-09-19T07:37:16.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.329+0000 7f969aa00640 1 -- 172.21.15.159:0/3885787156 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96940035f0 con 0x7f96a4103d20 2024-09-19T07:37:16.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.329+0000 7f96a8e00640 1 -- 172.21.15.159:0/3885787156 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a41a1ef0 con 0x7f96a4103d20 2024-09-19T07:37:16.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.329+0000 7f969aa00640 1 -- 172.21.15.159:0/3885787156 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694010690 con 0x7f96a4103d20 2024-09-19T07:37:16.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.329+0000 7f96a8e00640 1 -- 172.21.15.159:0/3885787156 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a41a2350 con 0x7f96a4103d20 2024-09-19T07:37:16.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.330+0000 7f96a8e00640 1 -- 172.21.15.159:0/3885787156 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a410b0c0 con 0x7f96a4103d20 2024-09-19T07:37:16.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.332+0000 7f969aa00640 1 -- 172.21.15.159:0/3885787156 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f96940037a0 con 0x7f96a4103d20 2024-09-19T07:37:16.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.334+0000 7f969aa00640 1 --2- 172.21.15.159:0/3885787156 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f96880a3250 0x7f96880a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:16.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.334+0000 7f969aa00640 1 -- 172.21.15.159:0/3885787156 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f96940d0330 con 0x7f96a4103d20 2024-09-19T07:37:16.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.334+0000 7f96a1400640 1 --2- 172.21.15.159:0/3885787156 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f96880a3250 0x7f96880a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:16.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.334+0000 7f96a1400640 1 --2- 172.21.15.159:0/3885787156 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f96880a3250 0x7f96880a5710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f96900095f0 tx=0x7f9690005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:16.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.336+0000 7f969aa00640 1 -- 172.21.15.159:0/3885787156 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f969409cc40 con 0x7f96a4103d20 2024-09-19T07:37:16.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.510+0000 7f96a8e00640 1 -- 172.21.15.159:0/3885787156 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f96a402da10 con 0x7f96a4103d20 2024-09-19T07:37:16.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.511+0000 7f969aa00640 1 -- 172.21.15.159:0/3885787156 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f969409c540 con 0x7f96a4103d20 2024-09-19T07:37:16.512 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-09-19T07:37:16.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.513+0000 7f96a8e00640 1 -- 172.21.15.159:0/3885787156 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f96880a3250 msgr2=0x7f96880a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:16.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.513+0000 7f96a8e00640 1 --2- 172.21.15.159:0/3885787156 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f96880a3250 0x7f96880a5710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f96900095f0 tx=0x7f9690005f90 comp rx=0 tx=0).stop 2024-09-19T07:37:16.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.513+0000 7f96a8e00640 1 -- 172.21.15.159:0/3885787156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96a4103d20 msgr2=0x7f96a4199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:16.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.513+0000 7f96a8e00640 1 --2- 172.21.15.159:0/3885787156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96a4103d20 0x7f96a4199fe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f969400e9c0 tx=0x7f969400ee80 comp rx=0 tx=0).stop 2024-09-19T07:37:16.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.513+0000 7f96a8e00640 1 -- 172.21.15.159:0/3885787156 shutdown_connections 2024-09-19T07:37:16.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.513+0000 7f96a8e00640 1 --2- 172.21.15.159:0/3885787156 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f96880a3250 0x7f96880a5710 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f96900095f0 tx=0x7f9690005f90 comp rx=0 tx=0).stop 2024-09-19T07:37:16.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.513+0000 7f96a8e00640 1 --2- 172.21.15.159:0/3885787156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a41058d0 0x7f96a41a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.513+0000 7f96a8e00640 1 --2- 172.21.15.159:0/3885787156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96a4104f10 0x7f96a419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.513+0000 7f96a8e00640 1 --2- 172.21.15.159:0/3885787156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96a4103d20 0x7f96a4199fe0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.513+0000 7f96a8e00640 1 -- 172.21.15.159:0/3885787156 >> 172.21.15.159:0/3885787156 conn(0x7f96a40ff4a0 msgr2=0x7f96a4108290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:16.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.513+0000 7f96a8e00640 1 -- 172.21.15.159:0/3885787156 shutdown_connections 2024-09-19T07:37:16.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.513+0000 7f96a8e00640 1 -- 172.21.15.159:0/3885787156 wait complete. 2024-09-19T07:37:16.530 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T07:37:16.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.688+0000 7feb17200640 1 -- 172.21.15.159:0/3995272549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb18102480 msgr2=0x7feb18104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:16.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.688+0000 7feb17200640 1 --2- 172.21.15.159:0/3995272549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb18102480 0x7feb18104870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feb08009900 tx=0x7feb0802a3f0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.688+0000 7feb17200640 1 -- 172.21.15.159:0/3995272549 shutdown_connections 2024-09-19T07:37:16.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.688+0000 7feb17200640 1 --2- 172.21.15.159:0/3995272549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb181076e0 0x7feb18109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.688+0000 7feb17200640 1 --2- 172.21.15.159:0/3995272549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb18104db0 0x7feb181071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.688+0000 7feb17200640 1 --2- 172.21.15.159:0/3995272549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb18102480 0x7feb18104870 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.688+0000 7feb17200640 1 -- 172.21.15.159:0/3995272549 >> 172.21.15.159:0/3995272549 conn(0x7feb180fc0a0 msgr2=0x7feb180fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:16.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.688+0000 7feb17200640 1 -- 172.21.15.159:0/3995272549 shutdown_connections 2024-09-19T07:37:16.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.688+0000 7feb17200640 1 -- 172.21.15.159:0/3995272549 wait complete. 2024-09-19T07:37:16.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.689+0000 7feb17200640 1 Processor -- start 2024-09-19T07:37:16.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.689+0000 7feb17200640 1 -- start start 2024-09-19T07:37:16.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.689+0000 7feb17200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb18102480 0x7feb18199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:16.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.689+0000 7feb17200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb18104db0 0x7feb1819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:16.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.689+0000 7feb17200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb181076e0 0x7feb181a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:16.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.689+0000 7feb17200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feb1810d730 con 0x7feb18102480 2024-09-19T07:37:16.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.689+0000 7feb17200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb1810d5b0 con 0x7feb181076e0 2024-09-19T07:37:16.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.689+0000 7feb17200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feb1810d8b0 con 0x7feb18104db0 2024-09-19T07:37:16.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.690+0000 7feb15e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb18102480 0x7feb18199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:16.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.690+0000 7feb16800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb181076e0 0x7feb181a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:16.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.690+0000 7feb15e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb18102480 0x7feb18199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58254/0 (socket says 172.21.15.159:58254) 2024-09-19T07:37:16.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.690+0000 7feb15e00640 1 -- 172.21.15.159:0/713948408 learned_addr learned my addr 172.21.15.159:0/713948408 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:37:16.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.690+0000 7feb15400640 1 --2- 172.21.15.159:0/713948408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb18104db0 0x7feb1819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:16.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.690+0000 7feb16800640 1 -- 172.21.15.159:0/713948408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb18104db0 msgr2=0x7feb1819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:16.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.690+0000 7feb16800640 1 --2- 172.21.15.159:0/713948408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb18104db0 0x7feb1819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.690+0000 7feb16800640 1 -- 172.21.15.159:0/713948408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb18102480 msgr2=0x7feb18199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:16.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.690+0000 7feb16800640 1 --2- 172.21.15.159:0/713948408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb18102480 0x7feb18199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.690+0000 7feb16800640 1 -- 172.21.15.159:0/713948408 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb181a1bd0 con 0x7feb181076e0 2024-09-19T07:37:16.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.690+0000 7feb16800640 1 --2- 172.21.15.159:0/713948408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb181076e0 0x7feb181a1560 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feb10005fd0 tx=0x7feb1000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:16.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.691+0000 7feb0ea00640 1 -- 172.21.15.159:0/713948408 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1000cdf0 con 0x7feb181076e0 2024-09-19T07:37:16.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.691+0000 7feb0ea00640 1 -- 172.21.15.159:0/713948408 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb10005020 con 0x7feb181076e0 2024-09-19T07:37:16.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.691+0000 7feb0ea00640 1 -- 172.21.15.159:0/713948408 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb100106d0 con 0x7feb181076e0 2024-09-19T07:37:16.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.691+0000 7feb17200640 1 -- 172.21.15.159:0/713948408 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb181a1ec0 con 0x7feb181076e0 2024-09-19T07:37:16.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.691+0000 7feb17200640 1 -- 172.21.15.159:0/713948408 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb181a2330 con 0x7feb181076e0 2024-09-19T07:37:16.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.692+0000 7feb17200640 1 -- 172.21.15.159:0/713948408 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb18109220 con 0x7feb181076e0 2024-09-19T07:37:16.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.694+0000 7feb0ea00640 1 -- 172.21.15.159:0/713948408 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7feb100054d0 con 0x7feb181076e0 2024-09-19T07:37:16.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.695+0000 7feb0ea00640 1 --2- 172.21.15.159:0/713948408 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feafc0a32d0 0x7feafc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:16.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.695+0000 7feb0ea00640 1 -- 172.21.15.159:0/713948408 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7feb100d0540 con 0x7feb181076e0 2024-09-19T07:37:16.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.696+0000 7feb15e00640 1 --2- 172.21.15.159:0/713948408 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feafc0a32d0 0x7feafc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:16.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.696+0000 7feb15e00640 1 --2- 172.21.15.159:0/713948408 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feafc0a32d0 0x7feafc0a5790 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feb080096c0 tx=0x7feb080315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:16.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.697+0000 7feb0ea00640 1 -- 172.21.15.159:0/713948408 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feb1009ce50 con 0x7feb181076e0 2024-09-19T07:37:16.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.872+0000 7feb17200640 1 -- 172.21.15.159:0/713948408 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feb1802da10 con 0x7feb181076e0 2024-09-19T07:37:16.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.873+0000 7feb0ea00640 1 -- 172.21.15.159:0/713948408 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7feb1009c750 con 0x7feb181076e0 2024-09-19T07:37:16.874 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-09-19T07:37:16.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.875+0000 7feb17200640 1 -- 172.21.15.159:0/713948408 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feafc0a32d0 msgr2=0x7feafc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:16.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.875+0000 7feb17200640 1 --2- 172.21.15.159:0/713948408 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feafc0a32d0 0x7feafc0a5790 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feb080096c0 tx=0x7feb080315b0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.875+0000 7feb17200640 1 -- 172.21.15.159:0/713948408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb181076e0 msgr2=0x7feb181a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:16.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.875+0000 7feb17200640 1 --2- 172.21.15.159:0/713948408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb181076e0 0x7feb181a1560 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feb10005fd0 tx=0x7feb1000ef10 comp rx=0 tx=0).stop 2024-09-19T07:37:16.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.875+0000 7feb17200640 1 -- 172.21.15.159:0/713948408 shutdown_connections 2024-09-19T07:37:16.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.875+0000 7feb17200640 1 --2- 172.21.15.159:0/713948408 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7feafc0a32d0 0x7feafc0a5790 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.875+0000 7feb17200640 1 --2- 172.21.15.159:0/713948408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb181076e0 0x7feb181a1560 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.875+0000 7feb17200640 1 --2- 172.21.15.159:0/713948408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb18104db0 0x7feb1819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.875+0000 7feb17200640 1 --2- 172.21.15.159:0/713948408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb18102480 0x7feb18199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.875+0000 7feb17200640 1 -- 172.21.15.159:0/713948408 >> 172.21.15.159:0/713948408 conn(0x7feb180fc0a0 msgr2=0x7feb18101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:16.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.875+0000 7feb17200640 1 -- 172.21.15.159:0/713948408 shutdown_connections 2024-09-19T07:37:16.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:16.875+0000 7feb17200640 1 -- 172.21.15.159:0/713948408 wait complete. 2024-09-19T07:37:16.892 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T07:37:17.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.050+0000 7f62bd000640 1 -- 172.21.15.159:0/1280464823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b81099a0 msgr2=0x7f62b810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.050+0000 7f62bd000640 1 --2- 172.21.15.159:0/1280464823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b81099a0 0x7f62b810be60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009ca0 tx=0x7f62b002a950 comp rx=0 tx=0).stop 2024-09-19T07:37:17.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.050+0000 7f62bd000640 1 -- 172.21.15.159:0/1280464823 shutdown_connections 2024-09-19T07:37:17.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.050+0000 7f62bd000640 1 --2- 172.21.15.159:0/1280464823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b81099a0 0x7f62b810be60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.050+0000 7f62bd000640 1 --2- 172.21.15.159:0/1280464823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b8074bf0 0x7f62b8075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.050+0000 7f62bd000640 1 --2- 172.21.15.159:0/1280464823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62b80761d0 0x7f62b8074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.050+0000 7f62bd000640 1 -- 172.21.15.159:0/1280464823 >> 172.21.15.159:0/1280464823 conn(0x7f62b80fd2b0 msgr2=0x7f62b80ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:17.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.051+0000 7f62bd000640 1 -- 172.21.15.159:0/1280464823 shutdown_connections 2024-09-19T07:37:17.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.051+0000 7f62bd000640 1 -- 172.21.15.159:0/1280464823 wait complete. 2024-09-19T07:37:17.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.051+0000 7f62bd000640 1 Processor -- start 2024-09-19T07:37:17.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.052+0000 7f62bd000640 1 -- start start 2024-09-19T07:37:17.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.052+0000 7f62bd000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b8074bf0 0x7f62b819a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:17.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.052+0000 7f62bd000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b80761d0 0x7f62b819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:17.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.052+0000 7f62bd000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62b81099a0 0x7f62b81a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:17.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.052+0000 7f62bd000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f62b810f9f0 con 0x7f62b8074bf0 2024-09-19T07:37:17.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.052+0000 7f62bd000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f62b810f870 con 0x7f62b81099a0 2024-09-19T07:37:17.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.052+0000 7f62bd000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f62b810fb70 con 0x7f62b80761d0 2024-09-19T07:37:17.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.052+0000 7f62b5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b8074bf0 0x7f62b819a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:17.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.052+0000 7f62b5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b8074bf0 0x7f62b819a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58268/0 (socket says 172.21.15.159:58268) 2024-09-19T07:37:17.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.052+0000 7f62b5e00640 1 -- 172.21.15.159:0/172004166 learned_addr learned my addr 172.21.15.159:0/172004166 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:37:17.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.052+0000 7f62afe00640 1 --2- 172.21.15.159:0/172004166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b80761d0 0x7f62b819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:17.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.052+0000 7f62b6800640 1 --2- 172.21.15.159:0/172004166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62b81099a0 0x7f62b81a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:17.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.052+0000 7f62b5e00640 1 -- 172.21.15.159:0/172004166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b80761d0 msgr2=0x7f62b819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.052+0000 7f62b5e00640 1 --2- 172.21.15.159:0/172004166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b80761d0 0x7f62b819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.052+0000 7f62b5e00640 1 -- 172.21.15.159:0/172004166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62b81099a0 msgr2=0x7f62b81a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.052+0000 7f62b5e00640 1 --2- 172.21.15.159:0/172004166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62b81099a0 0x7f62b81a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.052+0000 7f62b5e00640 1 -- 172.21.15.159:0/172004166 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b81a1c60 con 0x7f62b8074bf0 2024-09-19T07:37:17.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.053+0000 7f62b5e00640 1 --2- 172.21.15.159:0/172004166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b8074bf0 0x7f62b819a040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f62a800e9f0 tx=0x7f62a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:17.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.053+0000 7f62aea00640 1 -- 172.21.15.159:0/172004166 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a800cd80 con 0x7f62b8074bf0 2024-09-19T07:37:17.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.053+0000 7f62bd000640 1 -- 172.21.15.159:0/172004166 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b81a1f50 con 0x7f62b8074bf0 2024-09-19T07:37:17.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.053+0000 7f62aea00640 1 -- 172.21.15.159:0/172004166 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a8005500 con 0x7f62b8074bf0 2024-09-19T07:37:17.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.053+0000 7f62bd000640 1 -- 172.21.15.159:0/172004166 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b81a23b0 con 0x7f62b8074bf0 2024-09-19T07:37:17.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.053+0000 7f62aea00640 1 -- 172.21.15.159:0/172004166 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8018630 con 0x7f62b8074bf0 2024-09-19T07:37:17.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.054+0000 7f62bd000640 1 -- 172.21.15.159:0/172004166 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b8108760 con 0x7f62b8074bf0 2024-09-19T07:37:17.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.056+0000 7f62aea00640 1 -- 172.21.15.159:0/172004166 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f62a80046c0 con 0x7f62b8074bf0 2024-09-19T07:37:17.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.057+0000 7f62aea00640 1 --2- 172.21.15.159:0/172004166 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f629c0a3250 0x7f629c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:17.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.057+0000 7f62aea00640 1 -- 172.21.15.159:0/172004166 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f62a80d0200 con 0x7f62b8074bf0 2024-09-19T07:37:17.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.058+0000 7f62afe00640 1 --2- 172.21.15.159:0/172004166 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f629c0a3250 0x7f629c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:17.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.058+0000 7f62afe00640 1 --2- 172.21.15.159:0/172004166 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f629c0a3250 0x7f629c0a5710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f62a00095f0 tx=0x7f62a0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:17.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.061+0000 7f62aea00640 1 -- 172.21.15.159:0/172004166 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f62a809cb10 con 0x7f62b8074bf0 2024-09-19T07:37:17.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.236+0000 7f62bd000640 1 -- 172.21.15.159:0/172004166 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f62b8062f30 con 0x7f62b8074bf0 2024-09-19T07:37:17.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.237+0000 7f62aea00640 1 -- 172.21.15.159:0/172004166 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7f62a809c410 con 0x7f62b8074bf0 2024-09-19T07:37:17.238 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-09-19T07:37:17.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.239+0000 7f62bd000640 1 -- 172.21.15.159:0/172004166 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f629c0a3250 msgr2=0x7f629c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.239+0000 7f62bd000640 1 --2- 172.21.15.159:0/172004166 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f629c0a3250 0x7f629c0a5710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f62a00095f0 tx=0x7f62a0005f90 comp rx=0 tx=0).stop 2024-09-19T07:37:17.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.239+0000 7f62bd000640 1 -- 172.21.15.159:0/172004166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b8074bf0 msgr2=0x7f62b819a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.239+0000 7f62bd000640 1 --2- 172.21.15.159:0/172004166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b8074bf0 0x7f62b819a040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f62a800e9f0 tx=0x7f62a800edf0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.239+0000 7f62bd000640 1 -- 172.21.15.159:0/172004166 shutdown_connections 2024-09-19T07:37:17.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.239+0000 7f62bd000640 1 --2- 172.21.15.159:0/172004166 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f629c0a3250 0x7f629c0a5710 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.239+0000 7f62bd000640 1 --2- 172.21.15.159:0/172004166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62b81099a0 0x7f62b81a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.239+0000 7f62bd000640 1 --2- 172.21.15.159:0/172004166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b80761d0 0x7f62b819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.239+0000 7f62bd000640 1 --2- 172.21.15.159:0/172004166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b8074bf0 0x7f62b819a040 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.239+0000 7f62bd000640 1 -- 172.21.15.159:0/172004166 >> 172.21.15.159:0/172004166 conn(0x7f62b80fd2b0 msgr2=0x7f62b8105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:17.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.239+0000 7f62bd000640 1 -- 172.21.15.159:0/172004166 shutdown_connections 2024-09-19T07:37:17.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.239+0000 7f62bd000640 1 -- 172.21.15.159:0/172004166 wait complete. 2024-09-19T07:37:17.254 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:16 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/602334380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:37:17.254 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:16 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3885787156' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:37:17.254 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:16 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/713948408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T07:37:17.256 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-19T07:37:17.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:16 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/602334380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:37:17.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:16 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3885787156' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:37:17.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:16 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/713948408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T07:37:17.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:16 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/602334380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:37:17.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:16 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3885787156' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:37:17.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:16 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/713948408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T07:37:17.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.413+0000 7fb4af200640 1 -- 172.21.15.159:0/2457960708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a8103d20 msgr2=0x7fb4a8104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.413+0000 7fb4af200640 1 --2- 172.21.15.159:0/2457960708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a8103d20 0x7fb4a8104120 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb498009900 tx=0x7fb49802a410 comp rx=0 tx=0).stop 2024-09-19T07:37:17.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.413+0000 7fb4af200640 1 -- 172.21.15.159:0/2457960708 shutdown_connections 2024-09-19T07:37:17.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.413+0000 7fb4af200640 1 --2- 172.21.15.159:0/2457960708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4a81058d0 0x7fb4a810c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.413+0000 7fb4af200640 1 --2- 172.21.15.159:0/2457960708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4a8104f10 0x7fb4a8105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.413+0000 7fb4af200640 1 --2- 172.21.15.159:0/2457960708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a8103d20 0x7fb4a8104120 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.413+0000 7fb4af200640 1 -- 172.21.15.159:0/2457960708 >> 172.21.15.159:0/2457960708 conn(0x7fb4a80ff4a0 msgr2=0x7fb4a81018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:17.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.413+0000 7fb4af200640 1 -- 172.21.15.159:0/2457960708 shutdown_connections 2024-09-19T07:37:17.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.413+0000 7fb4af200640 1 -- 172.21.15.159:0/2457960708 wait complete. 2024-09-19T07:37:17.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.414+0000 7fb4af200640 1 Processor -- start 2024-09-19T07:37:17.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.414+0000 7fb4af200640 1 -- start start 2024-09-19T07:37:17.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.415+0000 7fb4af200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a8103d20 0x7fb4a819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:17.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.415+0000 7fb4af200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4a8104f10 0x7fb4a819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:17.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.415+0000 7fb4af200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4a81058d0 0x7fb4a81a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:17.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.415+0000 7fb4af200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a810f980 con 0x7fb4a81058d0 2024-09-19T07:37:17.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.415+0000 7fb4af200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a810f800 con 0x7fb4a8103d20 2024-09-19T07:37:17.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.415+0000 7fb4af200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a810fb00 con 0x7fb4a8104f10 2024-09-19T07:37:17.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.415+0000 7fb4ad200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4a81058d0 0x7fb4a81a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:17.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.415+0000 7fb4ad200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4a81058d0 0x7fb4a81a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58288/0 (socket says 172.21.15.159:58288) 2024-09-19T07:37:17.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.415+0000 7fb4ad200640 1 -- 172.21.15.159:0/3053366834 learned_addr learned my addr 172.21.15.159:0/3053366834 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:37:17.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.415+0000 7fb4a7400640 1 --2- 172.21.15.159:0/3053366834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4a8104f10 0x7fb4a819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:17.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.415+0000 7fb4a7e00640 1 --2- 172.21.15.159:0/3053366834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a8103d20 0x7fb4a819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:17.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.415+0000 7fb4ad200640 1 -- 172.21.15.159:0/3053366834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4a8104f10 msgr2=0x7fb4a819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.415+0000 7fb4ad200640 1 --2- 172.21.15.159:0/3053366834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4a8104f10 0x7fb4a819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.415+0000 7fb4ad200640 1 -- 172.21.15.159:0/3053366834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a8103d20 msgr2=0x7fb4a819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.415+0000 7fb4ad200640 1 --2- 172.21.15.159:0/3053366834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a8103d20 0x7fb4a819a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.415+0000 7fb4ad200640 1 -- 172.21.15.159:0/3053366834 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a81a1cb0 con 0x7fb4a81058d0 2024-09-19T07:37:17.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.416+0000 7fb4ad200640 1 --2- 172.21.15.159:0/3053366834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4a81058d0 0x7fb4a81a1640 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000eb10 tx=0x7fb4a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:17.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.416+0000 7fb4a4c00640 1 -- 172.21.15.159:0/3053366834 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0004b20 con 0x7fb4a81058d0 2024-09-19T07:37:17.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.416+0000 7fb4a4c00640 1 -- 172.21.15.159:0/3053366834 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a0004cc0 con 0x7fb4a81058d0 2024-09-19T07:37:17.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.416+0000 7fb4af200640 1 -- 172.21.15.159:0/3053366834 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a81a1fa0 con 0x7fb4a81058d0 2024-09-19T07:37:17.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.416+0000 7fb4a4c00640 1 -- 172.21.15.159:0/3053366834 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0010760 con 0x7fb4a81058d0 2024-09-19T07:37:17.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.416+0000 7fb4af200640 1 -- 172.21.15.159:0/3053366834 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a81a2410 con 0x7fb4a81058d0 2024-09-19T07:37:17.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.417+0000 7fb4af200640 1 -- 172.21.15.159:0/3053366834 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a810b150 con 0x7fb4a81058d0 2024-09-19T07:37:17.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.419+0000 7fb4a4c00640 1 -- 172.21.15.159:0/3053366834 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fb4a0010900 con 0x7fb4a81058d0 2024-09-19T07:37:17.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.420+0000 7fb4a4c00640 1 --2- 172.21.15.159:0/3053366834 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb48c0a3250 0x7fb48c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:17.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.420+0000 7fb4a4c00640 1 -- 172.21.15.159:0/3053366834 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fb4a00d03a0 con 0x7fb4a81058d0 2024-09-19T07:37:17.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.421+0000 7fb4a7e00640 1 --2- 172.21.15.159:0/3053366834 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb48c0a3250 0x7fb48c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:17.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.421+0000 7fb4a7e00640 1 --2- 172.21.15.159:0/3053366834 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb48c0a3250 0x7fb48c0a5710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb498005ab0 tx=0x7fb498000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:17.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.423+0000 7fb4a4c00640 1 -- 172.21.15.159:0/3053366834 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb4a0054210 con 0x7fb4a81058d0 2024-09-19T07:37:17.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.599+0000 7fb4af200640 1 -- 172.21.15.159:0/3053366834 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb4a8062df0 con 0x7fb4a81058d0 2024-09-19T07:37:17.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.599+0000 7fb4a4c00640 1 -- 172.21.15.159:0/3053366834 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v100) ==== 103+0+11 (secure 0 0 0) 0x7fb4a009cdb0 con 0x7fb4a81058d0 2024-09-19T07:37:17.600 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-09-19T07:37:17.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.601+0000 7fb4af200640 1 -- 172.21.15.159:0/3053366834 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb48c0a3250 msgr2=0x7fb48c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.601+0000 7fb4af200640 1 --2- 172.21.15.159:0/3053366834 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb48c0a3250 0x7fb48c0a5710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb498005ab0 tx=0x7fb498000bc0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.601+0000 7fb4af200640 1 -- 172.21.15.159:0/3053366834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4a81058d0 msgr2=0x7fb4a81a1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.601+0000 7fb4af200640 1 --2- 172.21.15.159:0/3053366834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4a81058d0 0x7fb4a81a1640 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000eb10 tx=0x7fb4a000c2d0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.602+0000 7fb4af200640 1 -- 172.21.15.159:0/3053366834 shutdown_connections 2024-09-19T07:37:17.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.602+0000 7fb4af200640 1 --2- 172.21.15.159:0/3053366834 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fb48c0a3250 0x7fb48c0a5710 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.602+0000 7fb4af200640 1 --2- 172.21.15.159:0/3053366834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4a81058d0 0x7fb4a81a1640 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.602+0000 7fb4af200640 1 --2- 172.21.15.159:0/3053366834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4a8104f10 0x7fb4a819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.602+0000 7fb4af200640 1 --2- 172.21.15.159:0/3053366834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a8103d20 0x7fb4a819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.602+0000 7fb4af200640 1 -- 172.21.15.159:0/3053366834 >> 172.21.15.159:0/3053366834 conn(0x7fb4a80ff4a0 msgr2=0x7fb4a8108320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:17.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.602+0000 7fb4af200640 1 -- 172.21.15.159:0/3053366834 shutdown_connections 2024-09-19T07:37:17.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.602+0000 7fb4af200640 1 -- 172.21.15.159:0/3053366834 wait complete. 2024-09-19T07:37:17.619 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T07:37:17.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.773+0000 7fa96e800640 1 -- 172.21.15.159:0/3293112833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa968076bb0 msgr2=0x7fa968076fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.773+0000 7fa96e800640 1 --2- 172.21.15.159:0/3293112833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa968076bb0 0x7fa968076fb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa958009900 tx=0x7fa95802a3f0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.773+0000 7fa96e800640 1 -- 172.21.15.159:0/3293112833 shutdown_connections 2024-09-19T07:37:17.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.773+0000 7fa96e800640 1 --2- 172.21.15.159:0/3293112833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa968078790 0x7fa968079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.773+0000 7fa96e800640 1 --2- 172.21.15.159:0/3293112833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa968077df0 0x7fa968078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.773+0000 7fa96e800640 1 --2- 172.21.15.159:0/3293112833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa968076bb0 0x7fa968076fb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.773+0000 7fa96e800640 1 -- 172.21.15.159:0/3293112833 >> 172.21.15.159:0/3293112833 conn(0x7fa9680ff4b0 msgr2=0x7fa9681018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:17.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.773+0000 7fa96e800640 1 -- 172.21.15.159:0/3293112833 shutdown_connections 2024-09-19T07:37:17.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.773+0000 7fa96e800640 1 -- 172.21.15.159:0/3293112833 wait complete. 2024-09-19T07:37:17.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.774+0000 7fa96e800640 1 Processor -- start 2024-09-19T07:37:17.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.774+0000 7fa96e800640 1 -- start start 2024-09-19T07:37:17.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.775+0000 7fa96e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa968076bb0 0x7fa96819e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:17.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.775+0000 7fa96e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa968077df0 0x7fa96819e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:17.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.775+0000 7fa96e800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa968078790 0x7fa9681a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:17.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.775+0000 7fa96e800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa968113e00 con 0x7fa968077df0 2024-09-19T07:37:17.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.775+0000 7fa96e800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa968113c80 con 0x7fa968076bb0 2024-09-19T07:37:17.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.775+0000 7fa96e800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa968113f80 con 0x7fa968078790 2024-09-19T07:37:17.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.775+0000 7fa966a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa968077df0 0x7fa96819e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:17.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.775+0000 7fa966a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa968077df0 0x7fa96819e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58296/0 (socket says 172.21.15.159:58296) 2024-09-19T07:37:17.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.775+0000 7fa966a00640 1 -- 172.21.15.159:0/340850915 learned_addr learned my addr 172.21.15.159:0/340850915 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:37:17.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.775+0000 7fa967e00640 1 --2- 172.21.15.159:0/340850915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa968078790 0x7fa9681a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:17.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.775+0000 7fa967400640 1 --2- 172.21.15.159:0/340850915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa968076bb0 0x7fa96819e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:17.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.775+0000 7fa966a00640 1 -- 172.21.15.159:0/340850915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa968078790 msgr2=0x7fa9681a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.775+0000 7fa966a00640 1 --2- 172.21.15.159:0/340850915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa968078790 0x7fa9681a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.775+0000 7fa966a00640 1 -- 172.21.15.159:0/340850915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa968076bb0 msgr2=0x7fa96819e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.775+0000 7fa966a00640 1 --2- 172.21.15.159:0/340850915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa968076bb0 0x7fa96819e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.775+0000 7fa966a00640 1 -- 172.21.15.159:0/340850915 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9681a60a0 con 0x7fa968077df0 2024-09-19T07:37:17.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.775+0000 7fa967e00640 1 --2- 172.21.15.159:0/340850915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa968078790 0x7fa9681a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:37:17.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.776+0000 7fa966a00640 1 --2- 172.21.15.159:0/340850915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa968077df0 0x7fa96819e9c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa95400ea60 tx=0x7fa95400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:17.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.776+0000 7fa95fe00640 1 -- 172.21.15.159:0/340850915 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95400cd20 con 0x7fa968077df0 2024-09-19T07:37:17.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.776+0000 7fa96e800640 1 -- 172.21.15.159:0/340850915 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9681a6390 con 0x7fa968077df0 2024-09-19T07:37:17.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.776+0000 7fa95fe00640 1 -- 172.21.15.159:0/340850915 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9540035f0 con 0x7fa968077df0 2024-09-19T07:37:17.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.776+0000 7fa95fe00640 1 -- 172.21.15.159:0/340850915 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954005220 con 0x7fa968077df0 2024-09-19T07:37:17.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.776+0000 7fa96e800640 1 -- 172.21.15.159:0/340850915 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9681a68e0 con 0x7fa968077df0 2024-09-19T07:37:17.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.777+0000 7fa96e800640 1 -- 172.21.15.159:0/340850915 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa968110830 con 0x7fa968077df0 2024-09-19T07:37:17.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.780+0000 7fa95fe00640 1 -- 172.21.15.159:0/340850915 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fa9540037a0 con 0x7fa968077df0 2024-09-19T07:37:17.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.781+0000 7fa95fe00640 1 --2- 172.21.15.159:0/340850915 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa94c0a3250 0x7fa94c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:17.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.781+0000 7fa95fe00640 1 -- 172.21.15.159:0/340850915 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fa9540cff50 con 0x7fa968077df0 2024-09-19T07:37:17.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.781+0000 7fa967400640 1 --2- 172.21.15.159:0/340850915 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa94c0a3250 0x7fa94c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:17.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.782+0000 7fa967400640 1 --2- 172.21.15.159:0/340850915 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa94c0a3250 0x7fa94c0a5710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa9580314e0 tx=0x7fa958000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:17.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:17.785+0000 7fa95fe00640 1 -- 172.21.15.159:0/340850915 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa95409c860 con 0x7fa968077df0 2024-09-19T07:37:18.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:18.011+0000 7fa96e800640 1 -- 172.21.15.159:0/340850915 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fa96802d9e0 con 0x7fa968077df0 2024-09-19T07:37:18.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:18.021+0000 7fa95fe00640 1 -- 172.21.15.159:0/340850915 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+742347 (secure 0 0 0) 0x7fa95409c160 con 0x7fa968077df0 2024-09-19T07:37:18.033 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:37:18.033 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-19T07:37:18.033 INFO:teuthology.orchestra.run.smithi159.stdout: "epoch": 18, 2024-09-19T07:37:18.033 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: "active_gid": 14148, 2024-09-19T07:37:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: "active_name": "x", 2024-09-19T07:37:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: "active_addrs": { 2024-09-19T07:37:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-19T07:37:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-19T07:37:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6800", 2024-09-19T07:37:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 4243905339 2024-09-19T07:37:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v1", 2024-09-19T07:37:18.034 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:6801", 2024-09-19T07:37:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 4243905339 2024-09-19T07:37:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-19T07:37:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "active_addr": "172.21.15.159:6801/4243905339", 2024-09-19T07:37:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "active_change": "2024-09-19T07:23:30.597046+0000", 2024-09-19T07:37:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T07:37:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "available": true, 2024-09-19T07:37:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "standbys": [ 2024-09-19T07:37:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "gid": 24106, 2024-09-19T07:37:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "z", 2024-09-19T07:37:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_features": 4540701547738038271, 2024-09-19T07:37:18.035 INFO:teuthology.orchestra.run.smithi159.stdout: "available_modules": [ 2024-09-19T07:37:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "alerts", 2024-09-19T07:37:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "interval": { 2024-09-19T07:37:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "interval", 2024-09-19T07:37:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "60", 2024-09-19T07:37:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T07:37:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.038 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.039 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.040 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.041 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_destination": { 2024-09-19T07:37:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_destination", 2024-09-19T07:37:18.042 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Email address to send alerts to", 2024-09-19T07:37:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_from_name": { 2024-09-19T07:37:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_from_name", 2024-09-19T07:37:18.043 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.044 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.044 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.044 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "Ceph", 2024-09-19T07:37:18.044 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.044 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.044 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.044 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Email From: name", 2024-09-19T07:37:18.044 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.044 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_host": { 2024-09-19T07:37:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_host", 2024-09-19T07:37:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "SMTP server", 2024-09-19T07:37:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.045 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_password": { 2024-09-19T07:37:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_password", 2024-09-19T07:37:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Password to authenticate with", 2024-09-19T07:37:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.046 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_port": { 2024-09-19T07:37:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_port", 2024-09-19T07:37:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "465", 2024-09-19T07:37:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "SMTP port", 2024-09-19T07:37:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.047 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_sender": { 2024-09-19T07:37:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_sender", 2024-09-19T07:37:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "SMTP envelope sender", 2024-09-19T07:37:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.048 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_ssl": { 2024-09-19T07:37:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_ssl", 2024-09-19T07:37:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T07:37:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.049 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_user": { 2024-09-19T07:37:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_user", 2024-09-19T07:37:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "User to authenticate as", 2024-09-19T07:37:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.050 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "balancer", 2024-09-19T07:37:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "active": { 2024-09-19T07:37:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "active", 2024-09-19T07:37:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.051 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T07:37:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "begin_time": { 2024-09-19T07:37:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "begin_time", 2024-09-19T07:37:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.052 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0000", 2024-09-19T07:37:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T07:37:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:37:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "begin_weekday": { 2024-09-19T07:37:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "begin_weekday", 2024-09-19T07:37:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "uint", 2024-09-19T07:37:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.053 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0", 2024-09-19T07:37:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "0", 2024-09-19T07:37:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "6", 2024-09-19T07:37:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T07:37:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:37:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_compat_max_iterations": { 2024-09-19T07:37:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "crush_compat_max_iterations", 2024-09-19T07:37:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "uint", 2024-09-19T07:37:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.054 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "25", 2024-09-19T07:37:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "1", 2024-09-19T07:37:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "250", 2024-09-19T07:37:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T07:37:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_compat_metrics": { 2024-09-19T07:37:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "crush_compat_metrics", 2024-09-19T07:37:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.055 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T07:37:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T07:37:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T07:37:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_compat_step": { 2024-09-19T07:37:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "crush_compat_step", 2024-09-19T07:37:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.056 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0.5", 2024-09-19T07:37:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "0.001", 2024-09-19T07:37:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "0.999", 2024-09-19T07:37:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "aggressiveness of optimization", 2024-09-19T07:37:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T07:37:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "end_time": { 2024-09-19T07:37:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "end_time", 2024-09-19T07:37:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.057 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "2359", 2024-09-19T07:37:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T07:37:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:37:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "end_weekday": { 2024-09-19T07:37:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "end_weekday", 2024-09-19T07:37:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "uint", 2024-09-19T07:37:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.058 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0", 2024-09-19T07:37:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "0", 2024-09-19T07:37:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "6", 2024-09-19T07:37:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T07:37:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:37:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.059 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.060 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.061 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.062 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.063 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: "min_score": { 2024-09-19T07:37:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "min_score", 2024-09-19T07:37:18.064 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.065 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.065 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.065 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0", 2024-09-19T07:37:18.065 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.065 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.065 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.065 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T07:37:18.065 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.065 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.065 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.065 INFO:teuthology.orchestra.run.smithi159.stdout: "mode": { 2024-09-19T07:37:18.065 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mode", 2024-09-19T07:37:18.065 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.066 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.066 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.066 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "upmap", 2024-09-19T07:37:18.066 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.066 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.066 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.066 INFO:teuthology.orchestra.run.smithi159.stdout: "crush-compat", 2024-09-19T07:37:18.066 INFO:teuthology.orchestra.run.smithi159.stdout: "none", 2024-09-19T07:37:18.066 INFO:teuthology.orchestra.run.smithi159.stdout: "read", 2024-09-19T07:37:18.066 INFO:teuthology.orchestra.run.smithi159.stdout: "upmap", 2024-09-19T07:37:18.066 INFO:teuthology.orchestra.run.smithi159.stdout: "upmap-read" 2024-09-19T07:37:18.066 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.066 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Balancer mode", 2024-09-19T07:37:18.067 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.067 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.067 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.067 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_ids": { 2024-09-19T07:37:18.067 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "pool_ids", 2024-09-19T07:37:18.067 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.067 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.067 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.067 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.067 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.067 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.067 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.067 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T07:37:18.068 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.068 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.068 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.068 INFO:teuthology.orchestra.run.smithi159.stdout: "sleep_interval": { 2024-09-19T07:37:18.068 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "sleep_interval", 2024-09-19T07:37:18.068 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.068 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.068 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.068 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "60", 2024-09-19T07:37:18.068 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.068 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.069 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.069 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T07:37:18.069 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.069 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.069 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.069 INFO:teuthology.orchestra.run.smithi159.stdout: "upmap_max_deviation": { 2024-09-19T07:37:18.069 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "upmap_max_deviation", 2024-09-19T07:37:18.069 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.069 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.069 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.069 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5", 2024-09-19T07:37:18.069 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "1", 2024-09-19T07:37:18.069 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.070 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.070 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T07:37:18.070 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T07:37:18.070 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.070 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.070 INFO:teuthology.orchestra.run.smithi159.stdout: "upmap_max_optimizations": { 2024-09-19T07:37:18.070 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "upmap_max_optimizations", 2024-09-19T07:37:18.070 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "uint", 2024-09-19T07:37:18.070 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.070 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.070 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:18.070 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.070 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.071 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.071 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T07:37:18.071 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.071 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.071 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.071 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.071 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.071 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.071 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "cephadm", 2024-09-19T07:37:18.071 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.071 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.072 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.072 INFO:teuthology.orchestra.run.smithi159.stdout: "agent_down_multiplier": { 2024-09-19T07:37:18.072 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "agent_down_multiplier", 2024-09-19T07:37:18.072 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.072 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.072 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.072 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "3.0", 2024-09-19T07:37:18.072 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.072 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.072 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.072 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T07:37:18.072 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.072 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.072 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.073 INFO:teuthology.orchestra.run.smithi159.stdout: "agent_refresh_rate": { 2024-09-19T07:37:18.073 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "agent_refresh_rate", 2024-09-19T07:37:18.073 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.073 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.073 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.073 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "20", 2024-09-19T07:37:18.073 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.073 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.073 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.073 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T07:37:18.073 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.073 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.074 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.074 INFO:teuthology.orchestra.run.smithi159.stdout: "agent_starting_port": { 2024-09-19T07:37:18.074 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "agent_starting_port", 2024-09-19T07:37:18.074 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.074 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.074 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.074 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "4721", 2024-09-19T07:37:18.074 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.074 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.074 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.074 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T07:37:18.074 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.075 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.075 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.075 INFO:teuthology.orchestra.run.smithi159.stdout: "allow_ptrace": { 2024-09-19T07:37:18.075 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "allow_ptrace", 2024-09-19T07:37:18.075 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.075 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.075 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.075 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.075 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.075 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.075 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.075 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T07:37:18.076 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T07:37:18.076 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.076 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.076 INFO:teuthology.orchestra.run.smithi159.stdout: "autotune_interval": { 2024-09-19T07:37:18.076 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "autotune_interval", 2024-09-19T07:37:18.076 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.076 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.076 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.076 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "600", 2024-09-19T07:37:18.076 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.076 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.076 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.076 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T07:37:18.077 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.077 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.077 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.077 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.077 INFO:teuthology.orchestra.run.smithi159.stdout: "autotune_memory_target_ratio": { 2024-09-19T07:37:18.077 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T07:37:18.077 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.077 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.077 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.077 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0.7", 2024-09-19T07:37:18.077 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.077 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.077 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.077 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T07:37:18.078 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.078 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.078 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.078 INFO:teuthology.orchestra.run.smithi159.stdout: "cephadm_log_destination": { 2024-09-19T07:37:18.078 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "cephadm_log_destination", 2024-09-19T07:37:18.078 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.078 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.078 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.078 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.078 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.078 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.078 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.079 INFO:teuthology.orchestra.run.smithi159.stdout: "file", 2024-09-19T07:37:18.079 INFO:teuthology.orchestra.run.smithi159.stdout: "file,syslog", 2024-09-19T07:37:18.079 INFO:teuthology.orchestra.run.smithi159.stdout: "syslog" 2024-09-19T07:37:18.079 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.079 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T07:37:18.079 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.079 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.079 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.079 INFO:teuthology.orchestra.run.smithi159.stdout: "cgroups_split": { 2024-09-19T07:37:18.079 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "cgroups_split", 2024-09-19T07:37:18.079 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.079 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.079 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.080 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.080 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.080 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.080 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.080 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T07:37:18.080 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.080 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.080 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.080 INFO:teuthology.orchestra.run.smithi159.stdout: "config_checks_enabled": { 2024-09-19T07:37:18.080 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "config_checks_enabled", 2024-09-19T07:37:18.080 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.080 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.080 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.081 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.081 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.081 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.081 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.081 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T07:37:18.081 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.081 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.081 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.081 INFO:teuthology.orchestra.run.smithi159.stdout: "config_dashboard": { 2024-09-19T07:37:18.081 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "config_dashboard", 2024-09-19T07:37:18.081 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.081 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.081 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.082 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.082 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.082 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.082 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.082 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T07:37:18.082 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.082 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.082 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.082 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_alertmanager": { 2024-09-19T07:37:18.082 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_alertmanager", 2024-09-19T07:37:18.082 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.082 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.082 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.083 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T07:37:18.083 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.083 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.083 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.083 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Prometheus container image", 2024-09-19T07:37:18.083 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.083 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.083 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.083 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_base": { 2024-09-19T07:37:18.083 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_base", 2024-09-19T07:37:18.083 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.083 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.083 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.084 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T07:37:18.084 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.084 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.084 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.084 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Container image name, without the tag", 2024-09-19T07:37:18.084 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.084 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.084 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.084 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_elasticsearch": { 2024-09-19T07:37:18.084 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_elasticsearch", 2024-09-19T07:37:18.084 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.084 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.085 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.085 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T07:37:18.085 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.085 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.085 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.085 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "elasticsearch container image", 2024-09-19T07:37:18.085 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.085 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.085 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.085 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_grafana": { 2024-09-19T07:37:18.085 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_grafana", 2024-09-19T07:37:18.085 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.085 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.085 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.086 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T07:37:18.086 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.086 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.086 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.086 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Prometheus container image", 2024-09-19T07:37:18.086 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.086 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.086 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.086 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_haproxy": { 2024-09-19T07:37:18.086 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_haproxy", 2024-09-19T07:37:18.086 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.086 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.087 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.087 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T07:37:18.087 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.087 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.087 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.087 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "HAproxy container image", 2024-09-19T07:37:18.087 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.087 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.087 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.087 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_jaeger_agent": { 2024-09-19T07:37:18.087 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_jaeger_agent", 2024-09-19T07:37:18.088 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.088 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.088 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.088 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T07:37:18.088 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.088 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.088 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.088 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Jaeger agent container image", 2024-09-19T07:37:18.088 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.088 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.088 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.088 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_jaeger_collector": { 2024-09-19T07:37:18.088 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_jaeger_collector", 2024-09-19T07:37:18.089 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.089 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.089 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.089 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T07:37:18.089 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.089 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.089 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.089 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Jaeger collector container image", 2024-09-19T07:37:18.089 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.089 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.089 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.089 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.089 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_jaeger_query": { 2024-09-19T07:37:18.090 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_jaeger_query", 2024-09-19T07:37:18.090 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.090 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.090 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.090 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T07:37:18.090 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.090 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.090 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.090 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Jaeger query container image", 2024-09-19T07:37:18.090 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.090 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.090 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.090 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_keepalived": { 2024-09-19T07:37:18.091 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_keepalived", 2024-09-19T07:37:18.091 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.091 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.091 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.091 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T07:37:18.091 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.091 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.091 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.091 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Keepalived container image", 2024-09-19T07:37:18.091 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.091 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.091 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.091 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_loki": { 2024-09-19T07:37:18.092 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_loki", 2024-09-19T07:37:18.092 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.092 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.092 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.092 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T07:37:18.092 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.092 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.092 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.092 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Loki container image", 2024-09-19T07:37:18.092 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.092 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.092 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.092 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.092 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_node_exporter": { 2024-09-19T07:37:18.093 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_node_exporter", 2024-09-19T07:37:18.093 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.093 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.093 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.093 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T07:37:18.093 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.093 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.093 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.093 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Prometheus container image", 2024-09-19T07:37:18.093 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.093 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.093 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.094 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_nvmeof": { 2024-09-19T07:37:18.094 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_nvmeof", 2024-09-19T07:37:18.094 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.094 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.094 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.094 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T07:37:18.094 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.094 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.094 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.094 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Nvme-of container image", 2024-09-19T07:37:18.094 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.094 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.094 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.095 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_prometheus": { 2024-09-19T07:37:18.095 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_prometheus", 2024-09-19T07:37:18.095 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.095 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.095 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.095 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T07:37:18.095 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.095 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.095 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.095 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Prometheus container image", 2024-09-19T07:37:18.095 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.095 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.095 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.096 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_promtail": { 2024-09-19T07:37:18.096 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_promtail", 2024-09-19T07:37:18.096 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.096 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.096 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.096 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T07:37:18.096 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.096 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.096 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.096 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Promtail container image", 2024-09-19T07:37:18.096 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.096 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.096 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.097 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_samba": { 2024-09-19T07:37:18.097 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_samba", 2024-09-19T07:37:18.097 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.097 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.097 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.097 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T07:37:18.097 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.097 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.097 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.097 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Samba/SMB container image", 2024-09-19T07:37:18.097 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.097 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.098 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.098 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_snmp_gateway": { 2024-09-19T07:37:18.098 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_snmp_gateway", 2024-09-19T07:37:18.098 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.098 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.098 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.098 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T07:37:18.098 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.098 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.098 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.098 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "SNMP Gateway container image", 2024-09-19T07:37:18.098 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.098 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.099 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.099 INFO:teuthology.orchestra.run.smithi159.stdout: "container_init": { 2024-09-19T07:37:18.099 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_init", 2024-09-19T07:37:18.099 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.099 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.099 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.099 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.099 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.099 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.099 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.099 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T07:37:18.099 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.099 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.100 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.100 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.100 INFO:teuthology.orchestra.run.smithi159.stdout: "daemon_cache_timeout": { 2024-09-19T07:37:18.100 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "daemon_cache_timeout", 2024-09-19T07:37:18.100 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.100 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.100 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.100 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "600", 2024-09-19T07:37:18.100 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.100 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.100 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.100 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T07:37:18.100 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.101 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.101 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.101 INFO:teuthology.orchestra.run.smithi159.stdout: "default_cephadm_command_timeout": { 2024-09-19T07:37:18.101 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T07:37:18.101 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.101 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.101 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.101 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "900", 2024-09-19T07:37:18.101 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.101 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.101 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.102 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T07:37:18.102 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.102 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.102 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.102 INFO:teuthology.orchestra.run.smithi159.stdout: "default_registry": { 2024-09-19T07:37:18.102 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "default_registry", 2024-09-19T07:37:18.102 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.102 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.102 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.102 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "docker.io", 2024-09-19T07:37:18.102 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.102 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.102 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.103 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T07:37:18.103 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.103 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.103 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.103 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.103 INFO:teuthology.orchestra.run.smithi159.stdout: "device_cache_timeout": { 2024-09-19T07:37:18.103 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "device_cache_timeout", 2024-09-19T07:37:18.103 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.103 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.103 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.103 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "1800", 2024-09-19T07:37:18.103 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.103 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.103 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.104 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "seconds to cache device inventory", 2024-09-19T07:37:18.104 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.104 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.104 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.104 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.104 INFO:teuthology.orchestra.run.smithi159.stdout: "device_enhanced_scan": { 2024-09-19T07:37:18.104 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "device_enhanced_scan", 2024-09-19T07:37:18.104 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.104 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.104 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.104 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.104 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.104 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.104 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.105 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T07:37:18.105 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.105 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.105 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.105 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.105 INFO:teuthology.orchestra.run.smithi159.stdout: "facts_cache_timeout": { 2024-09-19T07:37:18.105 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "facts_cache_timeout", 2024-09-19T07:37:18.105 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.105 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.105 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.105 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "60", 2024-09-19T07:37:18.105 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.105 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.105 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.106 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "seconds to cache host facts data", 2024-09-19T07:37:18.106 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.106 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.106 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.106 INFO:teuthology.orchestra.run.smithi159.stdout: "grafana_dashboards_path": { 2024-09-19T07:37:18.106 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "grafana_dashboards_path", 2024-09-19T07:37:18.106 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.106 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.106 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.106 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T07:37:18.106 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.106 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.106 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.107 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T07:37:18.107 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.107 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.107 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.107 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.107 INFO:teuthology.orchestra.run.smithi159.stdout: "host_check_interval": { 2024-09-19T07:37:18.107 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "host_check_interval", 2024-09-19T07:37:18.107 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.107 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.107 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.107 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "600", 2024-09-19T07:37:18.107 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.107 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.108 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.108 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how frequently to perform a host check", 2024-09-19T07:37:18.108 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.108 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.108 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.108 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.108 INFO:teuthology.orchestra.run.smithi159.stdout: "hw_monitoring": { 2024-09-19T07:37:18.108 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "hw_monitoring", 2024-09-19T07:37:18.109 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.109 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.109 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.109 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.109 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.109 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.109 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.109 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T07:37:18.109 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.109 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.109 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.109 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.109 INFO:teuthology.orchestra.run.smithi159.stdout: "inventory_list_all": { 2024-09-19T07:37:18.110 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "inventory_list_all", 2024-09-19T07:37:18.110 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.110 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.110 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.110 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.110 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.110 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.110 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.110 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T07:37:18.110 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.110 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.110 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.110 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.110 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.111 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.111 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.111 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.111 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.111 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.111 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.111 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.111 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.111 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.111 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.111 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.111 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.111 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.111 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.112 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.112 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.112 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.112 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.112 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.112 INFO:teuthology.orchestra.run.smithi159.stdout: "log_refresh_metadata": { 2024-09-19T07:37:18.112 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_refresh_metadata", 2024-09-19T07:37:18.112 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.112 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.112 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.112 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.113 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.113 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.113 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.113 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T07:37:18.113 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.113 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.113 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.113 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.113 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.113 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.113 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.113 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.113 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.114 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.114 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.114 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.114 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.114 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T07:37:18.114 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.114 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.114 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.114 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.114 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.114 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.114 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.114 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.115 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.115 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.115 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.115 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.115 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.115 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.115 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.115 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.115 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.115 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.115 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.115 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.115 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.116 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.116 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.116 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.116 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.116 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.116 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.116 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.116 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.116 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.116 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.116 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.116 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.117 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.117 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.117 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.117 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.117 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.117 INFO:teuthology.orchestra.run.smithi159.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T07:37:18.117 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T07:37:18.117 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.117 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.117 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.117 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.117 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.117 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.117 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.118 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T07:37:18.118 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.118 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.118 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.118 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.118 INFO:teuthology.orchestra.run.smithi159.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T07:37:18.118 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T07:37:18.118 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.118 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.118 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.118 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "*", 2024-09-19T07:37:18.118 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.118 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.118 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.119 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T07:37:18.119 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.119 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.119 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.119 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.119 INFO:teuthology.orchestra.run.smithi159.stdout: "max_count_per_host": { 2024-09-19T07:37:18.119 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "max_count_per_host", 2024-09-19T07:37:18.119 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.119 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.119 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.119 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:18.119 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.119 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.119 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.120 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "max number of daemons per service per host", 2024-09-19T07:37:18.120 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.120 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.120 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.120 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.120 INFO:teuthology.orchestra.run.smithi159.stdout: "max_osd_draining_count": { 2024-09-19T07:37:18.120 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "max_osd_draining_count", 2024-09-19T07:37:18.120 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.120 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.120 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.120 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:18.120 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.120 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.121 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.121 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T07:37:18.121 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.121 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.121 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.121 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.121 INFO:teuthology.orchestra.run.smithi159.stdout: "migration_current": { 2024-09-19T07:37:18.121 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "migration_current", 2024-09-19T07:37:18.121 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.121 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.121 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.121 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.121 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.121 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.122 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.122 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "internal - do not modify", 2024-09-19T07:37:18.122 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.122 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.122 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.122 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.122 INFO:teuthology.orchestra.run.smithi159.stdout: "mode": { 2024-09-19T07:37:18.122 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mode", 2024-09-19T07:37:18.122 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.122 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.122 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.122 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "root", 2024-09-19T07:37:18.122 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.122 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.123 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.123 INFO:teuthology.orchestra.run.smithi159.stdout: "cephadm-package", 2024-09-19T07:37:18.123 INFO:teuthology.orchestra.run.smithi159.stdout: "root" 2024-09-19T07:37:18.123 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.123 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T07:37:18.123 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.123 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.123 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.123 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.123 INFO:teuthology.orchestra.run.smithi159.stdout: "oob_default_addr": { 2024-09-19T07:37:18.123 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "oob_default_addr", 2024-09-19T07:37:18.123 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.123 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.123 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.123 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "169.254.1.1", 2024-09-19T07:37:18.124 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.124 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.124 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.124 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T07:37:18.124 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.124 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.124 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.124 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.124 INFO:teuthology.orchestra.run.smithi159.stdout: "prometheus_alerts_path": { 2024-09-19T07:37:18.124 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "prometheus_alerts_path", 2024-09-19T07:37:18.124 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.124 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.124 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.124 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T07:37:18.125 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.125 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.125 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.125 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T07:37:18.125 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.125 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.125 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.125 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.125 INFO:teuthology.orchestra.run.smithi159.stdout: "registry_insecure": { 2024-09-19T07:37:18.125 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "registry_insecure", 2024-09-19T07:37:18.125 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.125 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.125 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.125 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.126 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.126 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.126 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.126 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T07:37:18.126 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.126 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.126 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.126 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.126 INFO:teuthology.orchestra.run.smithi159.stdout: "registry_password": { 2024-09-19T07:37:18.126 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "registry_password", 2024-09-19T07:37:18.126 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.126 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.126 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.126 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.127 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.127 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.127 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.127 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T07:37:18.127 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.127 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.127 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.127 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.127 INFO:teuthology.orchestra.run.smithi159.stdout: "registry_url": { 2024-09-19T07:37:18.127 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "registry_url", 2024-09-19T07:37:18.127 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.127 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.127 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.127 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.128 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.128 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.128 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.128 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T07:37:18.128 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.128 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.128 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.128 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.128 INFO:teuthology.orchestra.run.smithi159.stdout: "registry_username": { 2024-09-19T07:37:18.128 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "registry_username", 2024-09-19T07:37:18.128 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.128 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.128 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.128 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.129 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.129 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.129 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.129 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T07:37:18.129 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.129 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.129 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.129 INFO:teuthology.orchestra.run.smithi159.stdout: "secure_monitoring_stack": { 2024-09-19T07:37:18.129 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "secure_monitoring_stack", 2024-09-19T07:37:18.129 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.129 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.129 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.130 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.130 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.130 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.130 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.130 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T07:37:18.130 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.130 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.130 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.130 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.130 INFO:teuthology.orchestra.run.smithi159.stdout: "service_discovery_port": { 2024-09-19T07:37:18.130 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "service_discovery_port", 2024-09-19T07:37:18.130 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.130 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.131 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.131 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "8765", 2024-09-19T07:37:18.131 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.131 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.131 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.131 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "cephadm service discovery port", 2024-09-19T07:37:18.131 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.131 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.131 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.132 INFO:teuthology.orchestra.run.smithi159.stdout: "ssh_config_file": { 2024-09-19T07:37:18.132 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ssh_config_file", 2024-09-19T07:37:18.132 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.132 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.132 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.132 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.132 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.132 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.132 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.132 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T07:37:18.132 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.132 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.132 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.133 INFO:teuthology.orchestra.run.smithi159.stdout: "use_agent": { 2024-09-19T07:37:18.133 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "use_agent", 2024-09-19T07:37:18.133 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.133 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.133 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.133 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.133 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.133 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.133 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.133 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T07:37:18.133 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.133 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.134 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.134 INFO:teuthology.orchestra.run.smithi159.stdout: "use_repo_digest": { 2024-09-19T07:37:18.134 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "use_repo_digest", 2024-09-19T07:37:18.134 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.134 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.134 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.134 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.134 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.134 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.134 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.134 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T07:37:18.134 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.135 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.135 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.135 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.135 INFO:teuthology.orchestra.run.smithi159.stdout: "warn_on_failed_host_check": { 2024-09-19T07:37:18.135 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "warn_on_failed_host_check", 2024-09-19T07:37:18.135 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.135 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.135 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.135 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.135 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.135 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.135 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.135 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T07:37:18.135 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.136 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.136 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.136 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.136 INFO:teuthology.orchestra.run.smithi159.stdout: "warn_on_stray_daemons": { 2024-09-19T07:37:18.136 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "warn_on_stray_daemons", 2024-09-19T07:37:18.136 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.136 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.136 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.136 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.136 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.136 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.136 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.136 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T07:37:18.136 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.137 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.137 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.137 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.137 INFO:teuthology.orchestra.run.smithi159.stdout: "warn_on_stray_hosts": { 2024-09-19T07:37:18.137 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "warn_on_stray_hosts", 2024-09-19T07:37:18.137 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.137 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.137 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.137 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.137 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.137 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.137 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.137 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T07:37:18.137 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.138 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.138 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.138 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.138 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.138 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.138 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.138 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "crash", 2024-09-19T07:37:18.138 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.138 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.138 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.138 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.138 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.138 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.139 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.139 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.139 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.139 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.139 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.139 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.139 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.139 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.139 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.139 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.139 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.139 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.139 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.139 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.140 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.140 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.140 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.140 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.140 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.140 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.140 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.140 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.140 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.140 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.140 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.140 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.140 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.141 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.141 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.141 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.141 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.141 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.141 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.141 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.141 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.141 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.141 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.141 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.141 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.142 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.142 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.142 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.142 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.142 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.142 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.142 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.142 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.142 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.142 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.142 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.142 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.143 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.143 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.143 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.143 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.143 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.143 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.143 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.143 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.143 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.143 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.143 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.143 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.143 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.144 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.144 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.144 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.144 INFO:teuthology.orchestra.run.smithi159.stdout: "retain_interval": { 2024-09-19T07:37:18.144 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "retain_interval", 2024-09-19T07:37:18.144 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.144 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.144 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.144 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "31536000", 2024-09-19T07:37:18.144 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.144 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.144 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.144 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T07:37:18.145 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.145 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.145 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.145 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.145 INFO:teuthology.orchestra.run.smithi159.stdout: "warn_recent_interval": { 2024-09-19T07:37:18.145 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "warn_recent_interval", 2024-09-19T07:37:18.145 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.145 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.145 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.145 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "1209600", 2024-09-19T07:37:18.145 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.145 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.145 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.146 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T07:37:18.146 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.146 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.146 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.146 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.146 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.146 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.146 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.146 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "dashboard", 2024-09-19T07:37:18.146 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.146 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.146 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.147 INFO:teuthology.orchestra.run.smithi159.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T07:37:18.147 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T07:37:18.147 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.147 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.147 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.147 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:18.147 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.147 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.147 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.147 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.147 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.147 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.147 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.148 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.148 INFO:teuthology.orchestra.run.smithi159.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T07:37:18.148 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T07:37:18.148 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.148 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.148 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.148 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.148 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.148 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.148 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.148 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.148 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.149 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.149 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.149 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.149 INFO:teuthology.orchestra.run.smithi159.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T07:37:18.149 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T07:37:18.149 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.149 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.149 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.149 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.150 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.150 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.150 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.150 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.150 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.150 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.150 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.150 INFO:teuthology.orchestra.run.smithi159.stdout: "AUDIT_API_ENABLED": { 2024-09-19T07:37:18.150 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T07:37:18.150 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.150 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.150 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.151 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.151 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.151 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.151 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.151 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.151 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.151 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.151 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.151 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.151 INFO:teuthology.orchestra.run.smithi159.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T07:37:18.151 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T07:37:18.151 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.151 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.151 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.152 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.152 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.152 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.152 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.152 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.152 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.152 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.152 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.152 INFO:teuthology.orchestra.run.smithi159.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T07:37:18.152 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T07:37:18.152 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.152 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.152 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.153 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.153 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.153 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.153 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.153 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.153 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.153 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.153 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.153 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T07:37:18.153 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T07:37:18.153 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.153 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.154 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.154 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.154 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.154 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.154 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.154 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.154 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.154 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.154 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.154 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.154 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T07:37:18.154 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T07:37:18.154 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.155 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.155 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.155 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.155 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.155 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.155 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.155 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.155 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.155 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.155 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.155 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.155 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T07:37:18.155 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T07:37:18.155 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.156 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.156 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.156 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.156 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.156 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.156 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.156 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.156 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.156 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.156 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.156 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.156 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T07:37:18.156 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T07:37:18.157 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.157 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.157 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.157 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.157 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.157 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.157 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.157 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.157 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.157 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.157 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.157 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.157 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T07:37:18.157 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T07:37:18.158 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.158 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.158 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.158 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.158 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.158 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.158 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.158 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.158 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.158 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.158 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.158 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.158 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T07:37:18.159 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T07:37:18.159 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.159 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.159 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.159 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.159 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.159 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.159 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.159 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.159 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.159 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.159 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.159 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.159 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T07:37:18.160 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T07:37:18.160 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.160 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.160 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.160 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.160 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.160 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.160 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.160 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.160 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.160 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.160 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.160 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.161 INFO:teuthology.orchestra.run.smithi159.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T07:37:18.161 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T07:37:18.161 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.161 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.161 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.161 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.161 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.161 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.161 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.161 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.161 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.161 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.161 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.162 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.162 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T07:37:18.162 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T07:37:18.162 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.162 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.162 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.162 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "admin", 2024-09-19T07:37:18.162 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.162 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.162 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.162 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.162 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.162 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.162 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.162 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.163 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T07:37:18.163 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T07:37:18.163 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.163 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.163 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.163 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.163 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.163 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.163 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.163 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.163 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.163 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.163 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.164 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.164 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_API_URL": { 2024-09-19T07:37:18.164 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_API_URL", 2024-09-19T07:37:18.164 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.164 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.164 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.164 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.164 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.164 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.164 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.164 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.164 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.164 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.164 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.165 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.165 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T07:37:18.165 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T07:37:18.165 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.165 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.165 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.165 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "admin", 2024-09-19T07:37:18.165 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.165 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.165 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.165 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.165 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.165 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.165 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.166 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.166 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T07:37:18.166 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T07:37:18.166 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.166 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.166 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.166 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.166 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.166 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.166 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.166 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.166 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.166 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.166 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.166 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.167 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T07:37:18.167 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T07:37:18.167 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.167 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.167 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.167 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.167 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.167 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.167 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.167 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.167 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.167 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.167 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.168 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.168 INFO:teuthology.orchestra.run.smithi159.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T07:37:18.168 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T07:37:18.168 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.168 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.168 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.168 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.168 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.168 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.168 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.168 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.168 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.168 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.168 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.169 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.169 INFO:teuthology.orchestra.run.smithi159.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T07:37:18.169 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T07:37:18.169 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.169 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.169 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.169 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.169 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.169 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.169 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.169 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.169 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.169 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.170 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.170 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.170 INFO:teuthology.orchestra.run.smithi159.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T07:37:18.170 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T07:37:18.170 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.170 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.170 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.170 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.170 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.170 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.170 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.170 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.170 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.170 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.171 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.171 INFO:teuthology.orchestra.run.smithi159.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T07:37:18.171 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T07:37:18.171 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.171 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.171 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.171 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.171 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.171 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.171 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.171 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.171 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.171 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.171 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.172 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.172 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T07:37:18.172 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T07:37:18.172 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.172 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.172 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.172 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.172 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.172 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.172 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.172 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.172 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.172 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.173 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.173 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.173 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T07:37:18.173 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T07:37:18.173 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.173 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.173 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.173 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.173 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.173 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.173 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.173 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.173 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.174 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.174 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.174 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.174 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T07:37:18.174 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T07:37:18.174 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.174 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.174 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.174 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.174 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.174 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.174 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.174 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.175 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.175 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.175 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.175 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.175 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T07:37:18.175 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T07:37:18.175 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.175 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.175 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.175 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.175 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.175 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.175 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.175 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.176 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.176 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.176 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.176 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.176 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T07:37:18.176 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T07:37:18.176 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.176 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.176 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.176 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.176 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.176 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.176 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.176 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.176 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.177 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.177 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.177 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.177 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T07:37:18.177 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T07:37:18.177 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.177 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.177 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.177 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.177 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.177 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.177 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.177 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.177 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.177 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.178 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.178 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.178 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T07:37:18.178 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T07:37:18.178 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.178 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.178 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.178 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.178 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.178 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.178 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.178 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.178 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.178 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.178 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.179 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.179 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T07:37:18.179 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T07:37:18.179 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.179 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.179 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.179 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.179 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.179 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.179 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.179 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.179 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.179 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.179 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.179 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.180 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T07:37:18.180 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T07:37:18.180 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.180 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.180 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.180 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T07:37:18.180 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.180 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.180 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.180 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.180 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.180 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.180 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.180 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.181 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T07:37:18.181 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T07:37:18.181 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.181 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.181 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.181 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:18.181 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.181 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.181 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.181 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.181 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.181 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.181 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.182 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.182 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T07:37:18.182 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T07:37:18.182 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.182 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.182 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.182 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "8", 2024-09-19T07:37:18.182 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.182 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.182 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.182 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.182 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.182 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.183 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.183 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.183 INFO:teuthology.orchestra.run.smithi159.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T07:37:18.183 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T07:37:18.183 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.183 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.183 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.183 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "45", 2024-09-19T07:37:18.183 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.183 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.183 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.183 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.183 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.184 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.184 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.184 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.184 INFO:teuthology.orchestra.run.smithi159.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T07:37:18.184 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T07:37:18.184 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.184 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.184 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.184 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.184 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.184 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.184 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.185 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.185 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.185 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.185 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.185 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.185 INFO:teuthology.orchestra.run.smithi159.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T07:37:18.185 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T07:37:18.185 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.185 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.185 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.185 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "admin", 2024-09-19T07:37:18.185 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.185 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.186 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.186 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.186 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.186 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.186 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.186 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.186 INFO:teuthology.orchestra.run.smithi159.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T07:37:18.186 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T07:37:18.186 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.186 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.186 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.186 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.186 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.186 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.187 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.187 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.187 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.187 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.187 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.187 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.187 INFO:teuthology.orchestra.run.smithi159.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T07:37:18.187 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T07:37:18.187 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.187 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.187 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.187 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.187 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.187 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.188 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.188 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.188 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.188 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.188 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.188 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.188 INFO:teuthology.orchestra.run.smithi159.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T07:37:18.188 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T07:37:18.188 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.188 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.188 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.188 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0", 2024-09-19T07:37:18.188 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.188 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.189 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.189 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.189 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.189 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.189 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.189 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.189 INFO:teuthology.orchestra.run.smithi159.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T07:37:18.189 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T07:37:18.189 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.189 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.189 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.189 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:18.189 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.189 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.190 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.190 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.190 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.190 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.190 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.190 INFO:teuthology.orchestra.run.smithi159.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T07:37:18.190 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T07:37:18.190 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.190 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.190 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.190 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5", 2024-09-19T07:37:18.190 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.190 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.190 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.191 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.191 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.191 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.191 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.191 INFO:teuthology.orchestra.run.smithi159.stdout: "cross_origin_url": { 2024-09-19T07:37:18.191 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "cross_origin_url", 2024-09-19T07:37:18.191 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.191 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.191 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.191 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.191 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.191 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.191 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.191 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.192 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.192 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.192 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.192 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.192 INFO:teuthology.orchestra.run.smithi159.stdout: "crt_file": { 2024-09-19T07:37:18.192 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "crt_file", 2024-09-19T07:37:18.192 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.192 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.192 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.192 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.192 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.192 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.192 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.192 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.192 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.193 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.193 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.193 INFO:teuthology.orchestra.run.smithi159.stdout: "debug": { 2024-09-19T07:37:18.193 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "debug", 2024-09-19T07:37:18.193 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.193 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.193 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.193 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.193 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.193 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.193 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.194 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Enable/disable debug options", 2024-09-19T07:37:18.194 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.194 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.194 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.194 INFO:teuthology.orchestra.run.smithi159.stdout: "jwt_token_ttl": { 2024-09-19T07:37:18.194 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "jwt_token_ttl", 2024-09-19T07:37:18.194 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.194 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.194 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.194 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "28800", 2024-09-19T07:37:18.194 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.194 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.194 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.194 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.195 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.195 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.195 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.195 INFO:teuthology.orchestra.run.smithi159.stdout: "key_file": { 2024-09-19T07:37:18.195 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "key_file", 2024-09-19T07:37:18.195 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.195 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.195 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.195 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.195 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.195 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.195 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.195 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.196 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.196 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.196 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.196 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.196 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.196 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.196 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.196 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.196 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.196 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.196 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.196 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.196 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.197 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.197 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.197 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.197 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.197 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.197 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.197 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.197 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.197 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.197 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.197 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.197 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.197 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.197 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.198 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.198 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.198 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.198 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.198 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.198 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.198 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.198 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.198 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.198 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.198 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.198 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.198 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.198 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.199 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.199 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.199 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.199 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.199 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.199 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.199 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.199 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.199 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.199 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.199 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.199 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.199 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.200 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.200 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.200 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.200 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.200 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.200 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.200 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.200 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.200 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.200 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.200 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.200 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.200 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.200 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.201 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.201 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.201 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.201 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.201 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.201 INFO:teuthology.orchestra.run.smithi159.stdout: "motd": { 2024-09-19T07:37:18.201 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "motd", 2024-09-19T07:37:18.201 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.201 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.201 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.201 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.201 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.201 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.201 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.202 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "The message of the day", 2024-09-19T07:37:18.202 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.202 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.202 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.202 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.202 INFO:teuthology.orchestra.run.smithi159.stdout: "redirect_resolve_ip_addr": { 2024-09-19T07:37:18.202 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T07:37:18.202 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.202 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.202 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.202 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.202 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.202 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.202 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.203 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.203 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.203 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.203 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.203 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.203 INFO:teuthology.orchestra.run.smithi159.stdout: "server_addr": { 2024-09-19T07:37:18.203 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_addr", 2024-09-19T07:37:18.203 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.203 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.203 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.203 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "::", 2024-09-19T07:37:18.203 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.203 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.203 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.203 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.204 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.204 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.204 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.204 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.204 INFO:teuthology.orchestra.run.smithi159.stdout: "server_port": { 2024-09-19T07:37:18.204 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_port", 2024-09-19T07:37:18.204 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.204 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.204 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.204 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "8080", 2024-09-19T07:37:18.204 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.204 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.204 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.205 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.205 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.205 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.205 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.205 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.205 INFO:teuthology.orchestra.run.smithi159.stdout: "ssl": { 2024-09-19T07:37:18.205 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ssl", 2024-09-19T07:37:18.205 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.205 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.205 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.205 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.205 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.205 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.206 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.206 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.206 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.206 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.206 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.206 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.206 INFO:teuthology.orchestra.run.smithi159.stdout: "ssl_server_port": { 2024-09-19T07:37:18.206 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ssl_server_port", 2024-09-19T07:37:18.206 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.206 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.206 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.206 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "8443", 2024-09-19T07:37:18.206 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.206 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.207 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.207 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.207 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.207 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.207 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.209 INFO:teuthology.orchestra.run.smithi159.stdout: "standby_behaviour": { 2024-09-19T07:37:18.209 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "standby_behaviour", 2024-09-19T07:37:18.209 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.209 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.209 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.209 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "redirect", 2024-09-19T07:37:18.209 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.209 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.209 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.210 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.210 INFO:teuthology.orchestra.run.smithi159.stdout: "redirect" 2024-09-19T07:37:18.210 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.210 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.210 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.210 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.210 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.210 INFO:teuthology.orchestra.run.smithi159.stdout: "standby_error_status_code": { 2024-09-19T07:37:18.210 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "standby_error_status_code", 2024-09-19T07:37:18.210 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.210 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.210 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.210 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "500", 2024-09-19T07:37:18.211 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "400", 2024-09-19T07:37:18.211 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "599", 2024-09-19T07:37:18.211 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.211 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.211 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.211 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.211 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.211 INFO:teuthology.orchestra.run.smithi159.stdout: "url_prefix": { 2024-09-19T07:37:18.211 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "url_prefix", 2024-09-19T07:37:18.211 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.211 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.211 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.211 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.212 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.212 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.212 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.212 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.212 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.212 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.212 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.212 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.212 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.212 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.212 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "devicehealth", 2024-09-19T07:37:18.212 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.213 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.213 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.213 INFO:teuthology.orchestra.run.smithi159.stdout: "enable_monitoring": { 2024-09-19T07:37:18.213 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "enable_monitoring", 2024-09-19T07:37:18.213 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.213 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.213 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.213 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.213 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.213 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.213 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.213 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "monitor device health metrics", 2024-09-19T07:37:18.213 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.213 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.214 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.214 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.214 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.214 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.214 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.214 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.214 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.214 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.214 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.214 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.214 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.214 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.215 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.215 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.215 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.215 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.215 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.215 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.215 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.215 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.215 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.215 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.215 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.215 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.215 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.216 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.216 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.216 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.216 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.216 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.216 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.216 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.216 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.216 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.216 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.216 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.216 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.216 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.217 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.217 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.217 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.217 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.217 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.217 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.217 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.217 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.217 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.217 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.217 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.217 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.217 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.217 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.218 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.218 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.218 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.218 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.218 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.218 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.218 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.218 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.218 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.218 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.218 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.218 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.218 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.218 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.219 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.219 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.219 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.219 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.219 INFO:teuthology.orchestra.run.smithi159.stdout: "mark_out_threshold": { 2024-09-19T07:37:18.219 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mark_out_threshold", 2024-09-19T07:37:18.219 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.219 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.219 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.219 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "2419200", 2024-09-19T07:37:18.219 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.219 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.219 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.220 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T07:37:18.220 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.220 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.220 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.220 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_name": { 2024-09-19T07:37:18.220 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "pool_name", 2024-09-19T07:37:18.220 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.220 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.220 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.220 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "device_health_metrics", 2024-09-19T07:37:18.220 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.220 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.220 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.221 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T07:37:18.221 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.221 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.221 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.221 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.221 INFO:teuthology.orchestra.run.smithi159.stdout: "retention_period": { 2024-09-19T07:37:18.221 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "retention_period", 2024-09-19T07:37:18.221 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.221 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.221 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.221 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "15552000", 2024-09-19T07:37:18.221 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.221 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.221 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.222 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how long to retain device health metrics", 2024-09-19T07:37:18.222 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.222 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.222 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.222 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.222 INFO:teuthology.orchestra.run.smithi159.stdout: "scrape_frequency": { 2024-09-19T07:37:18.222 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "scrape_frequency", 2024-09-19T07:37:18.222 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.222 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.222 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.222 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "86400", 2024-09-19T07:37:18.222 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.222 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.223 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.223 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T07:37:18.223 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.223 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.223 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.223 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.223 INFO:teuthology.orchestra.run.smithi159.stdout: "self_heal": { 2024-09-19T07:37:18.223 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "self_heal", 2024-09-19T07:37:18.223 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.223 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.223 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.223 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.223 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.223 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.224 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.224 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T07:37:18.224 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.224 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.224 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.224 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.224 INFO:teuthology.orchestra.run.smithi159.stdout: "sleep_interval": { 2024-09-19T07:37:18.224 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "sleep_interval", 2024-09-19T07:37:18.224 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.224 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.224 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.224 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "600", 2024-09-19T07:37:18.224 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.225 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.225 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.225 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T07:37:18.225 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.225 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.225 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.225 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.225 INFO:teuthology.orchestra.run.smithi159.stdout: "warn_threshold": { 2024-09-19T07:37:18.225 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "warn_threshold", 2024-09-19T07:37:18.225 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.225 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.225 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.225 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "7257600", 2024-09-19T07:37:18.226 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.226 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.226 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.226 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T07:37:18.226 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.226 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.226 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.226 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.226 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.226 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.226 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.226 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "diskprediction_local", 2024-09-19T07:37:18.226 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.226 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.226 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.227 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.227 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.227 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.227 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.227 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.227 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.227 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.227 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.227 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.227 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.227 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.227 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.227 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.227 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.228 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.228 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.228 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.228 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.228 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.228 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.228 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.228 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.228 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.228 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.228 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.228 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.228 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.229 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.229 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.229 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.229 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.229 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.229 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.229 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.229 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.229 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.229 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.229 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.229 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.229 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.229 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.230 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.230 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.230 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.230 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.230 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.230 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.230 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.230 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.230 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.230 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.230 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.230 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.230 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.230 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.231 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.231 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.231 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.231 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.231 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.231 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.231 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.231 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.231 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.232 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.232 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.232 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.232 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.232 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.232 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.232 INFO:teuthology.orchestra.run.smithi159.stdout: "predict_interval": { 2024-09-19T07:37:18.232 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "predict_interval", 2024-09-19T07:37:18.232 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.232 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.232 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.232 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "86400", 2024-09-19T07:37:18.232 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.233 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.233 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.233 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.233 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.233 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.233 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.233 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.233 INFO:teuthology.orchestra.run.smithi159.stdout: "predictor_model": { 2024-09-19T07:37:18.233 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "predictor_model", 2024-09-19T07:37:18.233 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.233 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.233 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.233 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "prophetstor", 2024-09-19T07:37:18.234 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.234 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.234 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.234 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.234 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.234 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.234 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.234 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.234 INFO:teuthology.orchestra.run.smithi159.stdout: "sleep_interval": { 2024-09-19T07:37:18.234 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "sleep_interval", 2024-09-19T07:37:18.234 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.234 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.234 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.235 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "600", 2024-09-19T07:37:18.235 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.235 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.235 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.235 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.235 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.235 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.235 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.235 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.235 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.235 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.235 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.235 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "influx", 2024-09-19T07:37:18.235 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": false, 2024-09-19T07:37:18.235 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "influxdb python module not found", 2024-09-19T07:37:18.236 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.236 INFO:teuthology.orchestra.run.smithi159.stdout: "batch_size": { 2024-09-19T07:37:18.236 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "batch_size", 2024-09-19T07:37:18.236 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.236 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.236 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.236 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5000", 2024-09-19T07:37:18.236 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.236 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.236 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.236 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T07:37:18.236 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.236 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.236 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.237 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.237 INFO:teuthology.orchestra.run.smithi159.stdout: "database": { 2024-09-19T07:37:18.237 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "database", 2024-09-19T07:37:18.237 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.237 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.237 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.237 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "ceph", 2024-09-19T07:37:18.237 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.237 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.237 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.237 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T07:37:18.237 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.238 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.238 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.238 INFO:teuthology.orchestra.run.smithi159.stdout: "hostname": { 2024-09-19T07:37:18.238 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "hostname", 2024-09-19T07:37:18.238 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.238 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.238 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.238 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.238 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.238 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.238 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.238 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "InfluxDB server hostname", 2024-09-19T07:37:18.239 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.239 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.239 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.239 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.239 INFO:teuthology.orchestra.run.smithi159.stdout: "interval": { 2024-09-19T07:37:18.239 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "interval", 2024-09-19T07:37:18.239 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.239 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.239 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.239 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "30", 2024-09-19T07:37:18.239 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "5", 2024-09-19T07:37:18.239 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.239 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.239 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T07:37:18.240 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.240 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.240 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.240 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.240 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.240 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.240 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.240 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.240 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.240 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.240 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.240 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.240 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.240 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.241 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.241 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.241 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.241 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.241 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.241 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.241 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.241 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.241 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.241 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.241 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.241 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.241 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.241 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.242 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.242 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.242 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.242 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.242 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.242 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.242 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.242 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.242 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.242 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.242 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.242 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.242 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.242 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.242 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.243 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.243 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.243 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.243 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.243 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.243 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.243 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.243 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.243 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.243 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.243 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.243 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.243 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.243 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.244 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.244 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.244 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.244 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.244 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.244 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.244 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.244 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.244 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.244 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.244 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.244 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.244 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.244 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.245 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.245 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.245 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.245 INFO:teuthology.orchestra.run.smithi159.stdout: "password": { 2024-09-19T07:37:18.245 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "password", 2024-09-19T07:37:18.245 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.245 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.245 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.245 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.245 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.245 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.245 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.245 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "password of InfluxDB server user", 2024-09-19T07:37:18.245 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.246 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.246 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.246 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.246 INFO:teuthology.orchestra.run.smithi159.stdout: "port": { 2024-09-19T07:37:18.246 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "port", 2024-09-19T07:37:18.246 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.246 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.246 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.246 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "8086", 2024-09-19T07:37:18.246 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.246 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.246 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.246 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "InfluxDB server port", 2024-09-19T07:37:18.246 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.247 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.247 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.247 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.247 INFO:teuthology.orchestra.run.smithi159.stdout: "ssl": { 2024-09-19T07:37:18.247 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ssl", 2024-09-19T07:37:18.247 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.247 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.247 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.247 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "false", 2024-09-19T07:37:18.247 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.247 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.247 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.247 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:37:18.247 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.247 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.248 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.248 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.248 INFO:teuthology.orchestra.run.smithi159.stdout: "threads": { 2024-09-19T07:37:18.248 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "threads", 2024-09-19T07:37:18.248 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.248 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.248 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.248 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5", 2024-09-19T07:37:18.248 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "1", 2024-09-19T07:37:18.248 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "32", 2024-09-19T07:37:18.248 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.248 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T07:37:18.248 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.249 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.249 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.249 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.249 INFO:teuthology.orchestra.run.smithi159.stdout: "username": { 2024-09-19T07:37:18.249 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "username", 2024-09-19T07:37:18.249 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.249 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.249 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.249 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.249 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.249 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.249 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.249 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "username of InfluxDB server user", 2024-09-19T07:37:18.249 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.250 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.250 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.250 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.250 INFO:teuthology.orchestra.run.smithi159.stdout: "verify_ssl": { 2024-09-19T07:37:18.250 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "verify_ssl", 2024-09-19T07:37:18.250 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.250 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.250 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.250 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "true", 2024-09-19T07:37:18.250 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.250 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.250 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.250 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:37:18.251 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.251 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.251 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.251 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.251 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.251 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.251 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.251 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "insights", 2024-09-19T07:37:18.251 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.251 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.251 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.251 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.251 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.252 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.252 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.252 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.252 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.252 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.252 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.252 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.252 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.252 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.252 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.252 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.252 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.252 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.253 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.253 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.253 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.253 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.253 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.253 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.253 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.253 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.253 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.253 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.253 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.253 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.253 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.253 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.254 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.254 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.254 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.254 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.254 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.254 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.254 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.254 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.254 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.254 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.254 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.254 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.254 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.255 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.255 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.255 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.255 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.255 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.255 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.255 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.255 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.255 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.255 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.255 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.255 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.255 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.255 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.256 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.256 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.256 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.256 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.256 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.256 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.256 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.256 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.256 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.256 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.256 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.256 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.256 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.256 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.257 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.257 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.257 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.257 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "iostat", 2024-09-19T07:37:18.257 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.257 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.257 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.257 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.257 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.257 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.257 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.257 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.257 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.258 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.258 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.258 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.258 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.258 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.258 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.258 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.258 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.258 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.258 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.258 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.258 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.258 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.258 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.259 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.259 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.259 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.259 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.259 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.259 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.259 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.259 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.259 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.259 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.259 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.259 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.259 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.259 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.260 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.260 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.260 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.260 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.260 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.260 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.260 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.260 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.260 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.260 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.260 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.260 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.260 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.261 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.261 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.261 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.261 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.261 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.261 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.261 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.261 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.261 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.261 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.261 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.261 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.261 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.261 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.262 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.262 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.262 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.262 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.262 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.262 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.262 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.262 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.262 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.262 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.262 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.262 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.262 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "k8sevents", 2024-09-19T07:37:18.263 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.263 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.263 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.263 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_event_retention_days": { 2024-09-19T07:37:18.263 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ceph_event_retention_days", 2024-09-19T07:37:18.263 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.263 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.263 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.263 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "7", 2024-09-19T07:37:18.263 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.263 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.263 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.263 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T07:37:18.263 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.263 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.264 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.264 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.264 INFO:teuthology.orchestra.run.smithi159.stdout: "config_check_secs": { 2024-09-19T07:37:18.264 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "config_check_secs", 2024-09-19T07:37:18.264 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.264 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.264 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.264 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:18.264 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "10", 2024-09-19T07:37:18.264 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.264 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.264 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T07:37:18.264 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.265 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.265 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.265 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.265 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.265 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.265 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.265 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.265 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.265 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.265 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.265 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.265 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.265 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.266 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.266 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.266 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.266 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.266 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.266 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.266 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.266 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.266 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.266 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.266 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.266 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.266 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.266 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.267 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.267 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.267 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.267 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.267 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.267 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.267 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.267 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.267 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.267 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.267 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.267 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.267 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.267 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.268 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.268 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.268 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.268 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.268 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.268 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.268 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.268 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.268 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.268 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.268 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.268 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.268 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.269 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.269 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.269 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.269 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.269 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.269 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.269 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.269 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.269 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.269 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.269 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.269 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.269 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.269 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.270 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.270 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.270 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.270 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.270 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.270 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.270 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.270 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.270 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "localpool", 2024-09-19T07:37:18.270 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.270 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.270 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.270 INFO:teuthology.orchestra.run.smithi159.stdout: "failure_domain": { 2024-09-19T07:37:18.271 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "failure_domain", 2024-09-19T07:37:18.271 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.271 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.271 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.271 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "host", 2024-09-19T07:37:18.271 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.271 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.271 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.271 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "failure domain for any created local pool", 2024-09-19T07:37:18.271 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T07:37:18.271 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.271 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.271 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.272 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.272 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.272 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.272 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.272 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.272 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.272 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.272 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.272 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.272 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.272 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.272 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.272 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.273 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.273 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.273 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.273 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.273 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.273 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.273 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.273 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.273 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.273 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.273 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.273 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.273 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.273 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.274 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.275 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.275 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.275 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.275 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.275 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.275 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.275 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.275 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.275 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.275 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.275 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.275 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.275 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.275 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.276 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.276 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.276 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.276 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.276 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.276 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.276 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.276 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.276 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.276 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.276 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.276 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.276 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.276 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.277 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.277 INFO:teuthology.orchestra.run.smithi159.stdout: "min_size": { 2024-09-19T07:37:18.277 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "min_size", 2024-09-19T07:37:18.277 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.277 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.277 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.277 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.277 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.277 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.277 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.277 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "default min_size for any created local pool", 2024-09-19T07:37:18.277 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T07:37:18.277 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.278 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.278 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.278 INFO:teuthology.orchestra.run.smithi159.stdout: "num_rep": { 2024-09-19T07:37:18.278 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "num_rep", 2024-09-19T07:37:18.278 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.278 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.278 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.278 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "3", 2024-09-19T07:37:18.278 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.278 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.278 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.278 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "default replica count for any created local pool", 2024-09-19T07:37:18.278 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.279 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.279 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.279 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num": { 2024-09-19T07:37:18.279 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "pg_num", 2024-09-19T07:37:18.279 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.279 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.279 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.279 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "128", 2024-09-19T07:37:18.279 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.279 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.279 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.279 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T07:37:18.279 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.280 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.280 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.280 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.280 INFO:teuthology.orchestra.run.smithi159.stdout: "prefix": { 2024-09-19T07:37:18.280 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "prefix", 2024-09-19T07:37:18.280 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.280 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.280 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.280 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.280 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.280 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.280 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.280 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "name prefix for any created local pool", 2024-09-19T07:37:18.281 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.281 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.281 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.281 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.281 INFO:teuthology.orchestra.run.smithi159.stdout: "subtree": { 2024-09-19T07:37:18.281 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "subtree", 2024-09-19T07:37:18.281 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.281 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.281 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.281 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "rack", 2024-09-19T07:37:18.281 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.281 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.281 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.281 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T07:37:18.282 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T07:37:18.282 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.282 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.282 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.282 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.282 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.282 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mds_autoscaler", 2024-09-19T07:37:18.282 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.282 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.282 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.282 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.282 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.282 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.283 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.283 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.283 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.283 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.283 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.283 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.283 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.283 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.283 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.283 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.283 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.283 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.284 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.284 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.284 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.284 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.284 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.284 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.284 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.284 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.284 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.284 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.284 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.284 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.284 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.285 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.285 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.285 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.285 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.285 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.285 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.285 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.285 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.285 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.285 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.285 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.285 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.286 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.286 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.286 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.286 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.286 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.286 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.286 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.286 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.286 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.286 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.286 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.286 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.286 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.286 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.287 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.287 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.287 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.287 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.287 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.287 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.287 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.287 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.287 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.287 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.287 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.287 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.287 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.287 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.288 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.288 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.288 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.288 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mirroring", 2024-09-19T07:37:18.288 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.288 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.288 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.288 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.288 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.288 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.288 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.288 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.288 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.289 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.289 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.289 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.289 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.289 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.289 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.289 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.289 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.289 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.289 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.289 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.289 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.289 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.289 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.290 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.290 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.290 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.290 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.290 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.290 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.290 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.290 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.290 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.290 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.290 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.290 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.291 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.291 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.291 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.291 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.291 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.291 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.291 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.291 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.291 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.291 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.291 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.291 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.291 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.292 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.292 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.292 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.292 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.292 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.292 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.292 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.292 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.292 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.292 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.292 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.292 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.292 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.293 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.293 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.293 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.293 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.293 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.293 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.293 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.293 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.293 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.293 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.293 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.293 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.293 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.294 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.294 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.294 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "nfs", 2024-09-19T07:37:18.294 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.294 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.294 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.294 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.294 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.294 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.294 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.294 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.294 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.295 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.295 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.295 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.295 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.295 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.295 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.295 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.295 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.295 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.295 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.295 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.295 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.295 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.296 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.296 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.296 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.296 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.296 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.296 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.296 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.296 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.296 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.296 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.296 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.296 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.297 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.297 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.297 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.297 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.297 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.297 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.297 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.297 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.297 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.297 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.297 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.297 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.298 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.298 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.298 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.298 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.298 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.298 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.298 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.298 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.298 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.298 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.298 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.298 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.298 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.299 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.299 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.299 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.299 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.299 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.299 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.299 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.299 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.299 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.299 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.299 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.299 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.300 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.300 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.300 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.300 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.300 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.300 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "orchestrator", 2024-09-19T07:37:18.300 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.300 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.300 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.300 INFO:teuthology.orchestra.run.smithi159.stdout: "fail_fs": { 2024-09-19T07:37:18.300 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "fail_fs", 2024-09-19T07:37:18.300 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.300 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.300 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.301 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.301 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.301 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.301 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.301 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T07:37:18.301 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.301 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.301 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.301 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.301 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.301 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.301 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.301 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.301 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.302 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.302 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.302 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.302 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.302 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.302 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.302 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.302 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.302 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.302 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.302 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.302 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.302 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.302 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.303 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.303 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.303 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.303 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.303 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.303 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.303 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.303 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.303 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.303 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.303 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.303 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.303 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.303 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.304 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.304 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.304 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.304 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.304 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.304 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.304 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.304 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.304 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.304 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.304 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.304 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.305 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.305 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.305 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.305 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.305 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.305 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.305 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.305 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.305 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.305 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.305 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.305 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.305 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.306 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.306 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.306 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.306 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.306 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.306 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.306 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.306 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.306 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.306 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.306 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.306 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.306 INFO:teuthology.orchestra.run.smithi159.stdout: "orchestrator": { 2024-09-19T07:37:18.307 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "orchestrator", 2024-09-19T07:37:18.307 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.307 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.307 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.307 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.307 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.307 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.307 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.307 INFO:teuthology.orchestra.run.smithi159.stdout: "cephadm", 2024-09-19T07:37:18.307 INFO:teuthology.orchestra.run.smithi159.stdout: "rook", 2024-09-19T07:37:18.307 INFO:teuthology.orchestra.run.smithi159.stdout: "test_orchestrator" 2024-09-19T07:37:18.307 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.307 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Orchestrator backend", 2024-09-19T07:37:18.308 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.308 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.308 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.308 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.308 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.308 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.308 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.308 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "osd_perf_query", 2024-09-19T07:37:18.308 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.308 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.308 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.308 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.309 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.309 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.309 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.309 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.309 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.309 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.309 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.309 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.309 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.309 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.309 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.309 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.309 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.309 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.310 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.310 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.310 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.310 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.310 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.310 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.310 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.310 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.310 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.310 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.310 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.310 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.310 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.311 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.311 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.311 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.311 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.311 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.311 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.311 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.311 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.311 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.311 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.311 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.311 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.312 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.312 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.312 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.312 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.312 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.312 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.312 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.312 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.312 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.312 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.312 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.312 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.312 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.312 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.313 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.313 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.313 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.313 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.313 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.313 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.313 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.313 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.313 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.313 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.313 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.313 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.313 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.314 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.314 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.314 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.314 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.314 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "osd_support", 2024-09-19T07:37:18.314 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.314 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.314 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.314 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.314 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.314 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.314 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.314 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.314 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.315 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.315 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.315 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.315 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.315 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.315 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.315 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.315 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.315 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.315 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.315 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.315 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.315 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.315 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.316 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.316 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.316 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.316 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.316 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.316 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.316 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.316 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.316 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.316 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.316 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.317 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.317 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.317 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.317 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.317 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.317 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.317 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.317 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.317 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.317 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.317 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.317 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.317 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.318 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.318 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.318 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.318 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.318 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.318 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.318 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.318 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.318 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.318 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.318 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.318 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.318 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.318 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.319 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.319 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.319 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.319 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.319 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.319 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.319 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.319 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.319 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.319 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.319 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.319 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.320 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.320 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.320 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "pg_autoscaler", 2024-09-19T07:37:18.320 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.320 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.320 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.320 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.320 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.320 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.320 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.320 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.320 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.320 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.321 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.321 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.321 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.321 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.321 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.321 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.321 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.321 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.321 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.321 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.321 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.321 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.321 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.321 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.322 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.322 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.322 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.322 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.322 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.322 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.322 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.322 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.322 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.322 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.322 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.322 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.322 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.322 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.322 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.323 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.323 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.323 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.323 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.323 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.323 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.323 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.323 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.323 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.323 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.323 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.323 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.324 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.324 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.324 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.324 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.324 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.324 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.324 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.324 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.324 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.325 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.325 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.325 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.325 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.325 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.325 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.325 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.325 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.325 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.325 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.325 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.325 INFO:teuthology.orchestra.run.smithi159.stdout: "sleep_interval": { 2024-09-19T07:37:18.325 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "sleep_interval", 2024-09-19T07:37:18.326 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.326 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.326 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.326 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "60", 2024-09-19T07:37:18.326 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.326 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.326 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.326 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.326 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.326 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.326 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.326 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.327 INFO:teuthology.orchestra.run.smithi159.stdout: "threshold": { 2024-09-19T07:37:18.327 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "threshold", 2024-09-19T07:37:18.327 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.327 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.327 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.327 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "3.0", 2024-09-19T07:37:18.327 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "1.0", 2024-09-19T07:37:18.327 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.327 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.327 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "scaling threshold", 2024-09-19T07:37:18.327 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T07:37:18.327 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.327 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.328 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.328 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.328 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.328 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "progress", 2024-09-19T07:37:18.328 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.328 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.328 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.328 INFO:teuthology.orchestra.run.smithi159.stdout: "allow_pg_recovery_event": { 2024-09-19T07:37:18.328 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "allow_pg_recovery_event", 2024-09-19T07:37:18.328 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.328 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.329 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.329 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.329 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.329 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.329 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.329 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T07:37:18.329 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.329 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.329 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.329 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.329 INFO:teuthology.orchestra.run.smithi159.stdout: "enabled": { 2024-09-19T07:37:18.329 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "enabled", 2024-09-19T07:37:18.329 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.329 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.330 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.330 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.330 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.330 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.330 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.330 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.330 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.330 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.330 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.330 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.330 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.330 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.331 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.331 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.331 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.331 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.331 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.331 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.331 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.331 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.331 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.332 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.332 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.332 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.332 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.332 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.332 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.332 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.332 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.332 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.332 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.332 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.332 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.332 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.333 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.333 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.333 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.333 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.333 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.333 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.333 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.333 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.333 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.333 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.333 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.333 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.333 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.334 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.334 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.334 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.334 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.334 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.334 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.334 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.334 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.334 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.334 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.334 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.334 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.334 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.334 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.334 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.335 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.335 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.335 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.335 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.335 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.335 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.335 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.335 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.335 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.335 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.335 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.335 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.335 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.335 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.335 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.336 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.336 INFO:teuthology.orchestra.run.smithi159.stdout: "max_completed_events": { 2024-09-19T07:37:18.336 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "max_completed_events", 2024-09-19T07:37:18.336 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.336 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.336 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.336 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "50", 2024-09-19T07:37:18.336 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.336 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.336 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.336 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "number of past completed events to remember", 2024-09-19T07:37:18.336 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.336 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.336 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.336 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.337 INFO:teuthology.orchestra.run.smithi159.stdout: "sleep_interval": { 2024-09-19T07:37:18.337 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "sleep_interval", 2024-09-19T07:37:18.337 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.337 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.337 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.337 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5", 2024-09-19T07:37:18.337 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.337 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.337 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.337 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how long the module is going to sleep", 2024-09-19T07:37:18.337 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.337 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.337 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.338 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.338 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.338 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.338 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "prometheus", 2024-09-19T07:37:18.338 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.338 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.338 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.338 INFO:teuthology.orchestra.run.smithi159.stdout: "cache": { 2024-09-19T07:37:18.338 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "cache", 2024-09-19T07:37:18.338 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.338 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.338 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.338 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.339 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.339 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.339 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.339 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.339 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.339 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.339 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.339 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.339 INFO:teuthology.orchestra.run.smithi159.stdout: "exclude_perf_counters": { 2024-09-19T07:37:18.339 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "exclude_perf_counters", 2024-09-19T07:37:18.339 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.339 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.340 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.340 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.340 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.340 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.340 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.340 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T07:37:18.340 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T07:37:18.340 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.340 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.340 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.340 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.340 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.340 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.341 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.341 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.341 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.341 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.341 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.341 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.341 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.341 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.341 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.341 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.341 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.341 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.341 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.342 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.342 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.342 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.342 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.342 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.342 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.342 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.342 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.342 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.342 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.342 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.342 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.342 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.343 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.343 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.343 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.343 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.343 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.343 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.343 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.343 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.343 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.343 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.343 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.343 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.343 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.343 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.344 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.344 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.344 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.344 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.344 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.344 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.344 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.344 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.344 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.344 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.344 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.344 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.344 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.344 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.345 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.345 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.345 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.345 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.345 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.345 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.345 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.345 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.345 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.345 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.345 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.345 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.345 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_stats_pools": { 2024-09-19T07:37:18.346 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rbd_stats_pools", 2024-09-19T07:37:18.346 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.346 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.346 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.346 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.346 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.346 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.346 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.346 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.346 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.346 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.346 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.346 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T07:37:18.347 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T07:37:18.347 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.347 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.347 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.347 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "300", 2024-09-19T07:37:18.347 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.347 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.347 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.347 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.347 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.347 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.347 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.348 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.348 INFO:teuthology.orchestra.run.smithi159.stdout: "scrape_interval": { 2024-09-19T07:37:18.348 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "scrape_interval", 2024-09-19T07:37:18.348 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.348 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.348 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.348 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "15.0", 2024-09-19T07:37:18.348 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.348 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.348 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.348 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.348 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.348 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.349 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.349 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.349 INFO:teuthology.orchestra.run.smithi159.stdout: "server_addr": { 2024-09-19T07:37:18.349 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_addr", 2024-09-19T07:37:18.349 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.349 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.349 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.349 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "::", 2024-09-19T07:37:18.349 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.349 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.349 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.349 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T07:37:18.349 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.350 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.350 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.350 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.350 INFO:teuthology.orchestra.run.smithi159.stdout: "server_port": { 2024-09-19T07:37:18.350 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_port", 2024-09-19T07:37:18.350 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.350 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.350 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.350 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "9283", 2024-09-19T07:37:18.350 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.350 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.350 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.350 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T07:37:18.350 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.350 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.351 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.351 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.351 INFO:teuthology.orchestra.run.smithi159.stdout: "stale_cache_strategy": { 2024-09-19T07:37:18.351 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "stale_cache_strategy", 2024-09-19T07:37:18.351 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.351 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.351 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.351 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "log", 2024-09-19T07:37:18.351 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.351 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.351 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.351 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.351 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.351 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.352 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.352 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.352 INFO:teuthology.orchestra.run.smithi159.stdout: "standby_behaviour": { 2024-09-19T07:37:18.352 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "standby_behaviour", 2024-09-19T07:37:18.352 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.352 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.352 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.352 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "default", 2024-09-19T07:37:18.352 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.352 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.352 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.352 INFO:teuthology.orchestra.run.smithi159.stdout: "default", 2024-09-19T07:37:18.352 INFO:teuthology.orchestra.run.smithi159.stdout: "error" 2024-09-19T07:37:18.353 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.353 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.353 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.353 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.353 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.353 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.353 INFO:teuthology.orchestra.run.smithi159.stdout: "standby_error_status_code": { 2024-09-19T07:37:18.353 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "standby_error_status_code", 2024-09-19T07:37:18.353 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.353 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.353 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.353 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "500", 2024-09-19T07:37:18.353 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "400", 2024-09-19T07:37:18.353 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "599", 2024-09-19T07:37:18.354 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.354 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.354 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.354 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.354 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.354 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.354 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.354 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.354 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rbd_support", 2024-09-19T07:37:18.354 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.354 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.354 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.355 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.355 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.355 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.355 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.355 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.355 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.355 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.355 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.355 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.355 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.355 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.355 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.355 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.356 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.356 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.356 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.356 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.356 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.356 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.356 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.356 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.356 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.356 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.356 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.356 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.356 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.356 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.356 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.357 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.357 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.357 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.357 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.357 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.357 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.357 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.357 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.357 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.357 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.357 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.357 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.357 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.358 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.358 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.358 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.358 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.358 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.358 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.358 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.358 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.358 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.358 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.358 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.358 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.358 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.358 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.358 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.359 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.359 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.359 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.359 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.359 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.359 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.359 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.359 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.359 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.359 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.359 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.359 INFO:teuthology.orchestra.run.smithi159.stdout: "max_concurrent_snap_create": { 2024-09-19T07:37:18.359 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "max_concurrent_snap_create", 2024-09-19T07:37:18.359 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.360 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.360 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.360 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:18.360 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.360 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.360 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.360 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.360 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.360 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.360 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.360 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.360 INFO:teuthology.orchestra.run.smithi159.stdout: "mirror_snapshot_schedule": { 2024-09-19T07:37:18.360 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T07:37:18.360 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.360 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.361 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.361 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.361 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.361 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.361 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.361 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.361 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.361 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.361 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.361 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.361 INFO:teuthology.orchestra.run.smithi159.stdout: "trash_purge_schedule": { 2024-09-19T07:37:18.362 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "trash_purge_schedule", 2024-09-19T07:37:18.362 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.362 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.362 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.362 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.362 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.362 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.362 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.362 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.362 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.362 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.362 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.362 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.362 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.362 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.363 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "restful", 2024-09-19T07:37:18.363 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.363 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.363 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.363 INFO:teuthology.orchestra.run.smithi159.stdout: "enable_auth": { 2024-09-19T07:37:18.363 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "enable_auth", 2024-09-19T07:37:18.363 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.363 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.363 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.363 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.363 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.363 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.363 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.363 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.364 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.364 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.364 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.364 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.364 INFO:teuthology.orchestra.run.smithi159.stdout: "key_file": { 2024-09-19T07:37:18.364 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "key_file", 2024-09-19T07:37:18.364 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.364 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.364 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.364 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.364 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.364 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.364 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.364 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.365 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.365 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.365 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.365 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.365 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.365 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.365 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.365 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.365 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.365 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.366 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.366 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.366 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.366 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.366 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.366 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.366 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.366 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.366 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.366 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.366 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.366 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.366 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.366 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.367 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.367 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.367 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.367 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.367 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.367 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.367 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.367 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.367 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.367 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.368 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.368 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.368 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.368 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.368 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.368 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.368 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.368 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.368 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.368 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.368 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.368 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.368 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.369 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.369 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.369 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.369 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.369 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.369 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.369 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.369 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.369 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.369 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.369 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.369 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.369 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.369 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.369 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.370 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.370 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.370 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.370 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.370 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.370 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.370 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.370 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.370 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.370 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.370 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.370 INFO:teuthology.orchestra.run.smithi159.stdout: "server_addr": { 2024-09-19T07:37:18.370 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_addr", 2024-09-19T07:37:18.370 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.371 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.371 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.371 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.371 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.371 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.371 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.371 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.371 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.371 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.371 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.371 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.371 INFO:teuthology.orchestra.run.smithi159.stdout: "server_port": { 2024-09-19T07:37:18.371 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_port", 2024-09-19T07:37:18.372 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.372 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.372 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.372 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.372 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.372 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.372 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.372 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.372 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.372 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.372 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.372 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.372 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.373 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.373 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rgw", 2024-09-19T07:37:18.373 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.373 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.373 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.373 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.373 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.373 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.373 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.373 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.373 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.373 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.374 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.374 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.374 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.374 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.374 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.374 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.374 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.374 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.374 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.374 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.374 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.374 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.374 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.375 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.375 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.375 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.375 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.375 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.375 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.375 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.375 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.375 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.375 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.375 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.375 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.376 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.376 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.376 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.376 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.376 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.376 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.376 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.376 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.376 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.376 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.376 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.376 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.376 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.376 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.377 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.377 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.377 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.377 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.377 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.377 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.377 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.377 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.377 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.377 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.377 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.377 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.377 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.377 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.378 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.378 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.378 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.378 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.378 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.378 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.378 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.378 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.378 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.378 INFO:teuthology.orchestra.run.smithi159.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T07:37:18.378 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T07:37:18.378 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.378 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.378 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.379 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5", 2024-09-19T07:37:18.379 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.379 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.379 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.379 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T07:37:18.379 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.379 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.379 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.379 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.379 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.379 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.379 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.379 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rook", 2024-09-19T07:37:18.379 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.380 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.380 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.380 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.380 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.380 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.380 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.380 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.380 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.380 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.380 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.380 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.380 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.380 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.381 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.381 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.381 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.381 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.381 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.381 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.381 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.381 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.381 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.381 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.381 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.381 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.381 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.382 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.382 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.382 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.382 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.382 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.382 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.382 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.382 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.382 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.382 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.382 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.382 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.382 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.383 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.383 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.383 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.383 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.383 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.383 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.383 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.383 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.383 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.383 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.383 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.383 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.384 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.384 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.384 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.384 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.384 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.384 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.384 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.384 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.384 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.384 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.385 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.385 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.385 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.385 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.385 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.385 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.385 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.385 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.385 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.385 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.385 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.385 INFO:teuthology.orchestra.run.smithi159.stdout: "storage_class": { 2024-09-19T07:37:18.385 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "storage_class", 2024-09-19T07:37:18.386 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.386 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.386 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.386 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "local", 2024-09-19T07:37:18.386 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.386 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.386 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.386 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T07:37:18.386 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.386 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.386 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.386 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.387 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.387 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.387 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.387 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "selftest", 2024-09-19T07:37:18.387 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.387 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.387 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.387 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.387 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.387 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.387 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.387 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.387 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.387 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.388 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.388 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.388 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.388 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.388 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.388 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.388 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.388 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.388 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.388 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.388 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.388 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.388 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.388 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.389 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.389 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.389 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.389 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.389 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.389 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.389 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.389 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.389 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.389 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.389 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.389 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.389 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.390 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.390 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.390 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.390 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.390 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.390 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.390 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.390 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.390 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.390 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.390 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.390 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.390 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.390 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.390 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.391 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.391 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.391 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.391 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.391 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.391 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.391 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.391 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.391 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.391 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.391 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.391 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.392 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.392 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.392 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.392 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.392 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.392 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.392 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.392 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.392 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.392 INFO:teuthology.orchestra.run.smithi159.stdout: "roption1": { 2024-09-19T07:37:18.392 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "roption1", 2024-09-19T07:37:18.392 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.392 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.392 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.392 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.393 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.393 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.393 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.393 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.393 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.393 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.393 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.393 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.393 INFO:teuthology.orchestra.run.smithi159.stdout: "roption2": { 2024-09-19T07:37:18.393 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "roption2", 2024-09-19T07:37:18.393 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.393 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.393 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.393 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "xyz", 2024-09-19T07:37:18.394 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.394 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.396 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.396 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.396 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.396 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.396 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.396 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.396 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption1": { 2024-09-19T07:37:18.396 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption1", 2024-09-19T07:37:18.396 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.396 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.397 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.397 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.397 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.397 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.397 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.397 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.397 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.397 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.397 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.397 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption2": { 2024-09-19T07:37:18.397 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption2", 2024-09-19T07:37:18.397 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.397 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.398 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.398 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.398 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.398 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.398 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.398 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.398 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.398 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.398 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.398 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.398 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption3": { 2024-09-19T07:37:18.398 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption3", 2024-09-19T07:37:18.398 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.398 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.399 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.399 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.399 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.399 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.399 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.399 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.399 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.399 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.399 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.399 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.399 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption4": { 2024-09-19T07:37:18.399 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption4", 2024-09-19T07:37:18.399 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.399 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.400 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.400 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.400 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.400 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.400 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.400 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.400 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.400 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.400 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.400 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.400 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption5": { 2024-09-19T07:37:18.400 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption5", 2024-09-19T07:37:18.400 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.400 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.400 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.401 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.401 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.401 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.401 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.401 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.401 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.401 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.401 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.401 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.401 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption6": { 2024-09-19T07:37:18.401 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption6", 2024-09-19T07:37:18.401 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.401 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.402 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.402 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.402 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.402 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.402 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.402 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.402 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.402 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.402 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.402 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.402 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption7": { 2024-09-19T07:37:18.402 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption7", 2024-09-19T07:37:18.402 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.402 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.402 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.403 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.403 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "1", 2024-09-19T07:37:18.403 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "42", 2024-09-19T07:37:18.403 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.403 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.403 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.403 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.403 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.403 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.403 INFO:teuthology.orchestra.run.smithi159.stdout: "testkey": { 2024-09-19T07:37:18.403 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "testkey", 2024-09-19T07:37:18.403 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.403 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.404 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.404 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.404 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.404 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.404 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.404 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.404 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.404 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.404 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.404 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.404 INFO:teuthology.orchestra.run.smithi159.stdout: "testlkey": { 2024-09-19T07:37:18.404 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "testlkey", 2024-09-19T07:37:18.404 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.405 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.405 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.405 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.405 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.405 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.405 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.405 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.405 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.405 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.405 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.405 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.405 INFO:teuthology.orchestra.run.smithi159.stdout: "testnewline": { 2024-09-19T07:37:18.405 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "testnewline", 2024-09-19T07:37:18.405 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.406 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.406 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.406 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.406 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.406 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.406 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.406 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.406 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.406 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.406 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.406 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.406 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.406 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.406 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.406 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "snap_schedule", 2024-09-19T07:37:18.406 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.407 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.407 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.407 INFO:teuthology.orchestra.run.smithi159.stdout: "allow_m_granularity": { 2024-09-19T07:37:18.407 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "allow_m_granularity", 2024-09-19T07:37:18.407 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.407 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.407 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.407 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.407 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.407 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.407 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.407 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T07:37:18.407 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.407 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.407 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.408 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.408 INFO:teuthology.orchestra.run.smithi159.stdout: "dump_on_update": { 2024-09-19T07:37:18.408 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "dump_on_update", 2024-09-19T07:37:18.408 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.408 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.408 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.408 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.408 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.408 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.408 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.408 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "dump database to debug log on update", 2024-09-19T07:37:18.408 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.408 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.408 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.409 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.409 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.409 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.409 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.409 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.409 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.409 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.409 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.409 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.409 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.409 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.409 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.409 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.409 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.410 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.410 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.410 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.410 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.410 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.410 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.410 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.410 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.410 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.410 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.410 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.410 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.410 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.410 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.411 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.411 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.411 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.411 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.411 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.411 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.411 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.411 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.411 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.411 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.411 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.411 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.411 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.412 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.412 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.412 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.412 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.412 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.412 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.412 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.412 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.412 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.412 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.412 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.412 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.412 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.413 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.413 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.413 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.413 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.413 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.413 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.413 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.413 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.413 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.413 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.413 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.413 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.413 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.414 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.414 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.414 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.414 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.414 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.414 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.414 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.414 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "stats", 2024-09-19T07:37:18.414 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.414 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.414 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.414 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.414 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.415 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.415 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.415 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.415 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.415 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.415 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.415 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.415 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.415 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.415 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.415 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.415 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.416 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.416 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.416 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.416 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.416 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.416 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.416 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.416 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.416 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.416 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.416 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.416 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.416 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.417 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.417 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.417 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.417 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.417 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.417 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.417 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.417 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.417 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.417 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.417 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.417 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.417 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.417 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.418 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.418 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.418 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.418 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.418 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.418 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.418 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.418 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.418 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.418 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.418 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.418 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.418 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.418 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.419 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.419 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.419 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.419 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.419 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.419 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.419 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.419 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.419 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.419 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.419 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.419 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.419 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.419 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.420 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.420 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.420 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.420 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.420 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "status", 2024-09-19T07:37:18.420 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.420 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.420 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.420 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.420 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.420 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.420 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.421 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.421 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.421 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.421 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.421 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.421 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.421 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.421 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.421 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.421 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.421 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.421 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.421 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.421 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.422 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.422 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.422 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.422 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.422 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.422 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.422 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.422 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.422 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.422 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.422 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.422 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.422 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.423 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.423 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.423 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.423 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.423 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.423 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.423 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.423 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.423 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.423 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.423 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.423 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.423 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.423 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.423 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.423 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.424 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.424 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.424 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.424 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.424 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.424 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.424 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.424 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.424 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.424 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.424 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.424 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.424 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.424 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.425 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.425 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.425 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.425 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.425 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.425 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.425 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.425 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.425 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.425 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.425 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.425 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.426 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "telegraf", 2024-09-19T07:37:18.426 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.426 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.426 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.426 INFO:teuthology.orchestra.run.smithi159.stdout: "address": { 2024-09-19T07:37:18.426 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "address", 2024-09-19T07:37:18.426 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.426 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.426 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.426 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T07:37:18.426 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.426 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.426 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.427 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.427 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.427 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.427 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.427 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.427 INFO:teuthology.orchestra.run.smithi159.stdout: "interval": { 2024-09-19T07:37:18.427 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "interval", 2024-09-19T07:37:18.427 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.427 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.427 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.427 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "15", 2024-09-19T07:37:18.427 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.427 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.428 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.428 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.428 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.428 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.428 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.428 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.428 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.428 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.428 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.428 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.428 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.428 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.428 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.429 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.429 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.429 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.429 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.429 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.429 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.429 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.429 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.429 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.429 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.429 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.429 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.429 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.429 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.430 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.430 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.430 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.430 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.430 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.430 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.430 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.430 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.430 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.430 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.430 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.430 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.430 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.431 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.431 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.431 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.431 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.431 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.431 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.431 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.431 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.431 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.431 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.431 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.431 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.432 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.432 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.432 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.432 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.432 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.432 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.432 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.432 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.432 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.432 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.433 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.433 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.433 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.433 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.433 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.433 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.433 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.433 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.433 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.433 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.433 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.433 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.433 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.433 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.434 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.434 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.434 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.434 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "telemetry", 2024-09-19T07:37:18.434 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.434 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.434 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.434 INFO:teuthology.orchestra.run.smithi159.stdout: "channel_basic": { 2024-09-19T07:37:18.434 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "channel_basic", 2024-09-19T07:37:18.434 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.434 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.434 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.434 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.435 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.435 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.435 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.435 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T07:37:18.435 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.435 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.435 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.435 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.435 INFO:teuthology.orchestra.run.smithi159.stdout: "channel_crash": { 2024-09-19T07:37:18.435 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "channel_crash", 2024-09-19T07:37:18.435 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.435 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.436 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.436 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.436 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.436 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.436 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.436 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T07:37:18.436 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.436 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.436 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.436 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.436 INFO:teuthology.orchestra.run.smithi159.stdout: "channel_device": { 2024-09-19T07:37:18.436 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "channel_device", 2024-09-19T07:37:18.436 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.437 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.437 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.437 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.437 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.437 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.437 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.437 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T07:37:18.437 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.437 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.437 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.437 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.437 INFO:teuthology.orchestra.run.smithi159.stdout: "channel_ident": { 2024-09-19T07:37:18.437 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "channel_ident", 2024-09-19T07:37:18.438 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.438 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.438 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.438 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.438 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.438 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.438 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.438 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T07:37:18.438 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.438 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.438 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.438 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.438 INFO:teuthology.orchestra.run.smithi159.stdout: "channel_perf": { 2024-09-19T07:37:18.438 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "channel_perf", 2024-09-19T07:37:18.438 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.439 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.439 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.439 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.439 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.439 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.439 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.439 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T07:37:18.439 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.439 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.439 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.439 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.439 INFO:teuthology.orchestra.run.smithi159.stdout: "contact": { 2024-09-19T07:37:18.439 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "contact", 2024-09-19T07:37:18.440 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.440 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.440 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.440 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.440 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.440 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.440 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.440 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.440 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.440 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.440 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.440 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.441 INFO:teuthology.orchestra.run.smithi159.stdout: "description": { 2024-09-19T07:37:18.441 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "description", 2024-09-19T07:37:18.441 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.441 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.441 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.441 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.441 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.441 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.441 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.441 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.441 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.441 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.441 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.441 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.442 INFO:teuthology.orchestra.run.smithi159.stdout: "device_url": { 2024-09-19T07:37:18.442 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "device_url", 2024-09-19T07:37:18.442 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.442 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.442 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.442 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T07:37:18.442 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.442 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.442 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.442 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.442 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.442 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.442 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.443 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.443 INFO:teuthology.orchestra.run.smithi159.stdout: "enabled": { 2024-09-19T07:37:18.443 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "enabled", 2024-09-19T07:37:18.443 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.443 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.443 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.443 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.443 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.443 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.443 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.443 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.443 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.443 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.444 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.444 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.444 INFO:teuthology.orchestra.run.smithi159.stdout: "interval": { 2024-09-19T07:37:18.444 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "interval", 2024-09-19T07:37:18.444 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.444 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.444 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.444 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "24", 2024-09-19T07:37:18.444 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "8", 2024-09-19T07:37:18.444 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.444 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.444 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.444 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.444 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.445 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.445 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.445 INFO:teuthology.orchestra.run.smithi159.stdout: "last_opt_revision": { 2024-09-19T07:37:18.445 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "last_opt_revision", 2024-09-19T07:37:18.445 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.445 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.445 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.445 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "1", 2024-09-19T07:37:18.445 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.445 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.445 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.445 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.446 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.446 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.446 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.446 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.446 INFO:teuthology.orchestra.run.smithi159.stdout: "leaderboard": { 2024-09-19T07:37:18.446 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "leaderboard", 2024-09-19T07:37:18.446 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.446 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.446 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.446 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.446 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.446 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.446 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.446 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.446 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.447 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.447 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.447 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.447 INFO:teuthology.orchestra.run.smithi159.stdout: "leaderboard_description": { 2024-09-19T07:37:18.447 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "leaderboard_description", 2024-09-19T07:37:18.447 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.447 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.447 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.447 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.447 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.447 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.447 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.447 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.447 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.447 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.448 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.448 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.448 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.448 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.448 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.448 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.448 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.448 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.448 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.448 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.448 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.448 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.448 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.448 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.449 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.449 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.449 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.449 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.449 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.449 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.449 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.449 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.449 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.449 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.449 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.449 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.449 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.449 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.450 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.450 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.450 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.450 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.450 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.450 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.450 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.450 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.450 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.450 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.450 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.450 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.450 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.450 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.451 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.451 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.451 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.451 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.451 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.451 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.451 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.451 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.451 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.451 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.451 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.451 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.451 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.451 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.452 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.452 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.452 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.452 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.452 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.452 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.452 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.452 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.452 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.452 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.452 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.452 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.452 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.452 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.453 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.453 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.453 INFO:teuthology.orchestra.run.smithi159.stdout: "organization": { 2024-09-19T07:37:18.453 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "organization", 2024-09-19T07:37:18.453 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.453 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.453 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.453 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.453 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.453 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.453 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.453 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.453 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.453 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.454 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.454 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.454 INFO:teuthology.orchestra.run.smithi159.stdout: "proxy": { 2024-09-19T07:37:18.454 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "proxy", 2024-09-19T07:37:18.454 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.454 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.454 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.454 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.454 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.454 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.454 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.454 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.454 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.454 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.455 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.455 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.455 INFO:teuthology.orchestra.run.smithi159.stdout: "url": { 2024-09-19T07:37:18.455 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "url", 2024-09-19T07:37:18.455 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.455 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.455 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.455 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T07:37:18.455 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.455 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.455 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.455 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.455 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.456 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.456 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.456 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.456 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.456 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.456 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.456 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "test_orchestrator", 2024-09-19T07:37:18.456 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.456 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.456 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.456 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.456 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.457 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.457 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.457 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.457 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.457 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.457 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.457 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.457 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.457 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.457 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.457 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.457 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.457 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.457 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.458 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.458 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.458 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.458 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.458 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.458 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.458 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.458 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.458 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.458 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.458 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.458 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.458 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.459 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.459 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.459 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.459 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.459 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.459 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.459 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.459 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.459 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.459 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.459 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.459 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.459 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.459 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.460 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.460 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.460 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.460 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.460 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.460 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.460 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.460 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.460 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.460 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.460 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.460 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.460 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.461 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.461 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.461 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.461 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.461 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.461 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.461 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.461 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.461 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.461 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.461 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.461 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.461 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.462 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.462 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.462 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.462 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.462 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "volumes", 2024-09-19T07:37:18.462 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.462 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.462 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.462 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.462 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.462 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.462 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.462 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.462 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.462 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.463 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.463 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.463 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.463 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.463 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.463 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.463 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.463 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.463 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.463 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.463 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.463 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.463 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.464 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.464 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.464 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.464 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.464 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.464 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.464 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.464 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.464 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.464 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.464 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.464 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.464 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.465 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.465 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.465 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.465 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.465 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.465 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.465 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.465 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.465 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.465 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.465 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.465 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.465 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.466 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.466 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.466 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.466 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.466 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.466 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.466 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.466 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.466 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.466 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.466 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.466 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.467 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.467 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.467 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.467 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.467 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.467 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.467 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.467 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.467 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.467 INFO:teuthology.orchestra.run.smithi159.stdout: "max_concurrent_clones": { 2024-09-19T07:37:18.467 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "max_concurrent_clones", 2024-09-19T07:37:18.467 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.467 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.467 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.468 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "4", 2024-09-19T07:37:18.468 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.468 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.468 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.468 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T07:37:18.468 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.468 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.468 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.468 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.468 INFO:teuthology.orchestra.run.smithi159.stdout: "periodic_async_work": { 2024-09-19T07:37:18.468 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "periodic_async_work", 2024-09-19T07:37:18.469 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.469 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.469 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.469 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.469 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.469 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.469 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.469 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Periodically check for async work", 2024-09-19T07:37:18.469 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.469 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.469 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.469 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.469 INFO:teuthology.orchestra.run.smithi159.stdout: "snapshot_clone_delay": { 2024-09-19T07:37:18.469 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "snapshot_clone_delay", 2024-09-19T07:37:18.470 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.470 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.470 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.470 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0", 2024-09-19T07:37:18.470 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.470 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.470 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.470 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T07:37:18.470 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.470 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.470 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.470 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.470 INFO:teuthology.orchestra.run.smithi159.stdout: "snapshot_clone_no_wait": { 2024-09-19T07:37:18.470 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T07:37:18.471 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.471 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.471 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.471 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.471 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.471 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.471 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.471 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T07:37:18.471 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.471 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.471 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.471 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.471 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.472 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.472 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.472 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "zabbix", 2024-09-19T07:37:18.472 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.472 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.472 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.472 INFO:teuthology.orchestra.run.smithi159.stdout: "discovery_interval": { 2024-09-19T07:37:18.472 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "discovery_interval", 2024-09-19T07:37:18.472 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "uint", 2024-09-19T07:37:18.472 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.472 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.472 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "100", 2024-09-19T07:37:18.472 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.473 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.473 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.473 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.473 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.473 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.473 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.473 INFO:teuthology.orchestra.run.smithi159.stdout: "identifier": { 2024-09-19T07:37:18.473 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "identifier", 2024-09-19T07:37:18.473 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.473 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.473 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.473 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.473 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.474 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.474 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.474 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.474 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.474 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.474 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.474 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.474 INFO:teuthology.orchestra.run.smithi159.stdout: "interval": { 2024-09-19T07:37:18.474 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "interval", 2024-09-19T07:37:18.474 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.474 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.474 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.474 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "60", 2024-09-19T07:37:18.475 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.475 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.475 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.475 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.475 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.475 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.475 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.475 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.475 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.475 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.475 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.475 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.475 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.476 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.476 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.476 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.476 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.476 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.476 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.476 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.476 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.476 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.476 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.476 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.476 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.476 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.477 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.477 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.477 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.477 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.477 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.477 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.477 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.477 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.477 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.477 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.477 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.477 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.477 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.478 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.478 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.478 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.478 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.478 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.478 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.478 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.478 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.478 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.478 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.478 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.478 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.479 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.479 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.479 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.479 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.479 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.479 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.479 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.479 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.479 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.479 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.479 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.479 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.480 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.480 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.480 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.480 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.480 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.480 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.480 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.480 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.480 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.480 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.480 INFO:teuthology.orchestra.run.smithi159.stdout: "zabbix_host": { 2024-09-19T07:37:18.480 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "zabbix_host", 2024-09-19T07:37:18.481 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.481 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.481 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.481 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.481 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.481 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.481 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.481 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.481 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.481 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.481 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.481 INFO:teuthology.orchestra.run.smithi159.stdout: "zabbix_port": { 2024-09-19T07:37:18.481 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "zabbix_port", 2024-09-19T07:37:18.482 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.482 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.482 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.482 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10051", 2024-09-19T07:37:18.482 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.482 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.482 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.482 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.482 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.482 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.482 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.482 INFO:teuthology.orchestra.run.smithi159.stdout: "zabbix_sender": { 2024-09-19T07:37:18.482 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "zabbix_sender", 2024-09-19T07:37:18.483 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.483 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.483 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.483 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T07:37:18.483 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.483 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.483 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.483 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.483 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.483 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.483 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.483 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.483 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.483 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.483 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-19T07:37:18.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.484 INFO:teuthology.orchestra.run.smithi159.stdout: "gid": 24116, 2024-09-19T07:37:18.484 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "y", 2024-09-19T07:37:18.484 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr_features": 4540701547738038271, 2024-09-19T07:37:18.484 INFO:teuthology.orchestra.run.smithi159.stdout: "available_modules": [ 2024-09-19T07:37:18.484 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.484 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "alerts", 2024-09-19T07:37:18.484 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.484 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.484 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.484 INFO:teuthology.orchestra.run.smithi159.stdout: "interval": { 2024-09-19T07:37:18.484 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "interval", 2024-09-19T07:37:18.485 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.485 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.485 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.485 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "60", 2024-09-19T07:37:18.485 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.485 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.485 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.485 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T07:37:18.485 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.485 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.485 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.485 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.486 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.486 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.486 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.486 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.486 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.486 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.486 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.486 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.486 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.486 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.486 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.486 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.486 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.487 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.487 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.487 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.487 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.487 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.487 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.487 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.487 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.487 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.487 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.487 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.487 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.487 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.488 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.488 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.488 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.488 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.488 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.488 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.488 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.488 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.488 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.488 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.488 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.488 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.488 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.489 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.489 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.489 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.489 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.489 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.489 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.489 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.489 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.489 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.489 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.489 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.489 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.489 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.490 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.490 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.490 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.490 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.490 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.490 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.490 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.490 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.490 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.490 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.490 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.490 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.490 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.490 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.491 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_destination": { 2024-09-19T07:37:18.491 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_destination", 2024-09-19T07:37:18.491 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.491 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.491 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.491 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.491 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.491 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.491 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.491 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Email address to send alerts to", 2024-09-19T07:37:18.491 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.491 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.491 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.491 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.491 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_from_name": { 2024-09-19T07:37:18.492 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_from_name", 2024-09-19T07:37:18.492 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.492 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.492 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.492 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "Ceph", 2024-09-19T07:37:18.492 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.492 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.492 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.492 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Email From: name", 2024-09-19T07:37:18.492 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.492 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.492 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.492 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.492 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_host": { 2024-09-19T07:37:18.492 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_host", 2024-09-19T07:37:18.493 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.493 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.493 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.493 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.493 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.493 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.493 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.493 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "SMTP server", 2024-09-19T07:37:18.493 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.493 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.493 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.493 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.493 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_password": { 2024-09-19T07:37:18.494 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_password", 2024-09-19T07:37:18.494 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.494 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.494 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.494 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.494 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.494 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.494 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.494 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Password to authenticate with", 2024-09-19T07:37:18.494 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.494 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.494 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.495 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_port": { 2024-09-19T07:37:18.495 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_port", 2024-09-19T07:37:18.495 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.495 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.495 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.495 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "465", 2024-09-19T07:37:18.495 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.495 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.495 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.495 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "SMTP port", 2024-09-19T07:37:18.495 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.495 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.495 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.495 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_sender": { 2024-09-19T07:37:18.496 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_sender", 2024-09-19T07:37:18.496 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.496 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.496 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.496 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.496 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.496 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.496 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.496 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "SMTP envelope sender", 2024-09-19T07:37:18.496 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.496 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.496 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.496 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.497 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_ssl": { 2024-09-19T07:37:18.497 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_ssl", 2024-09-19T07:37:18.497 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.497 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.497 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.497 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.497 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.497 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.497 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.497 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T07:37:18.497 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.497 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.497 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.497 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.498 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_user": { 2024-09-19T07:37:18.498 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_user", 2024-09-19T07:37:18.498 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.498 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.498 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.498 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.498 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.498 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.498 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.498 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "User to authenticate as", 2024-09-19T07:37:18.498 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.498 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.498 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.499 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.499 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.499 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.499 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "balancer", 2024-09-19T07:37:18.499 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.499 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.499 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.499 INFO:teuthology.orchestra.run.smithi159.stdout: "active": { 2024-09-19T07:37:18.499 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "active", 2024-09-19T07:37:18.499 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.499 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.499 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.500 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.500 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.500 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.500 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.500 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T07:37:18.500 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.500 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.500 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.500 INFO:teuthology.orchestra.run.smithi159.stdout: "begin_time": { 2024-09-19T07:37:18.500 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "begin_time", 2024-09-19T07:37:18.500 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.500 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.501 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.501 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0000", 2024-09-19T07:37:18.501 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.501 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.501 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.501 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T07:37:18.501 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:37:18.501 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.501 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.501 INFO:teuthology.orchestra.run.smithi159.stdout: "begin_weekday": { 2024-09-19T07:37:18.501 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "begin_weekday", 2024-09-19T07:37:18.501 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "uint", 2024-09-19T07:37:18.502 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.502 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.502 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0", 2024-09-19T07:37:18.502 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "0", 2024-09-19T07:37:18.502 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "6", 2024-09-19T07:37:18.502 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.502 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T07:37:18.502 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:37:18.502 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.502 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.502 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.502 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_compat_max_iterations": { 2024-09-19T07:37:18.502 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "crush_compat_max_iterations", 2024-09-19T07:37:18.503 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "uint", 2024-09-19T07:37:18.503 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.503 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.503 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "25", 2024-09-19T07:37:18.503 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "1", 2024-09-19T07:37:18.503 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "250", 2024-09-19T07:37:18.503 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.503 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T07:37:18.503 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.503 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.503 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.503 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_compat_metrics": { 2024-09-19T07:37:18.504 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "crush_compat_metrics", 2024-09-19T07:37:18.504 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.504 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.504 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.504 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T07:37:18.504 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.504 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.504 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.504 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T07:37:18.504 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T07:37:18.504 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.504 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.505 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_compat_step": { 2024-09-19T07:37:18.505 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "crush_compat_step", 2024-09-19T07:37:18.505 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.505 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.505 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.505 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0.5", 2024-09-19T07:37:18.505 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "0.001", 2024-09-19T07:37:18.505 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "0.999", 2024-09-19T07:37:18.505 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.505 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "aggressiveness of optimization", 2024-09-19T07:37:18.505 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T07:37:18.505 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.506 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.506 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.506 INFO:teuthology.orchestra.run.smithi159.stdout: "end_time": { 2024-09-19T07:37:18.506 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "end_time", 2024-09-19T07:37:18.506 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.506 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.506 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.506 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "2359", 2024-09-19T07:37:18.506 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.506 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.506 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.506 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T07:37:18.506 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:37:18.506 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.506 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.507 INFO:teuthology.orchestra.run.smithi159.stdout: "end_weekday": { 2024-09-19T07:37:18.507 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "end_weekday", 2024-09-19T07:37:18.507 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "uint", 2024-09-19T07:37:18.507 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.507 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.507 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0", 2024-09-19T07:37:18.507 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "0", 2024-09-19T07:37:18.507 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "6", 2024-09-19T07:37:18.507 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.507 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T07:37:18.507 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:37:18.507 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.507 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.507 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.508 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.508 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.508 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.508 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.508 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.508 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.508 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.508 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.508 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.508 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.508 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.508 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.508 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.509 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.509 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.509 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.509 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.509 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.509 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.509 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.509 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.509 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.509 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.509 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.509 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.509 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.510 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.510 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.510 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.510 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.510 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.510 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.510 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.510 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.510 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.510 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.510 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.510 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.510 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.510 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.511 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.511 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.511 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.511 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.511 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.511 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.511 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.511 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.511 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.511 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.511 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.511 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.512 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.512 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.512 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.512 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.512 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.512 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.512 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.512 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.512 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.512 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.512 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.513 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.513 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.513 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.513 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.513 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.513 INFO:teuthology.orchestra.run.smithi159.stdout: "min_score": { 2024-09-19T07:37:18.513 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "min_score", 2024-09-19T07:37:18.513 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.513 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.513 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.513 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0", 2024-09-19T07:37:18.513 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.514 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.514 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.514 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T07:37:18.514 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.514 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.514 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.514 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.514 INFO:teuthology.orchestra.run.smithi159.stdout: "mode": { 2024-09-19T07:37:18.514 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mode", 2024-09-19T07:37:18.514 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.514 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.514 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.514 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "upmap", 2024-09-19T07:37:18.515 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.515 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.515 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.515 INFO:teuthology.orchestra.run.smithi159.stdout: "crush-compat", 2024-09-19T07:37:18.515 INFO:teuthology.orchestra.run.smithi159.stdout: "none", 2024-09-19T07:37:18.515 INFO:teuthology.orchestra.run.smithi159.stdout: "read", 2024-09-19T07:37:18.515 INFO:teuthology.orchestra.run.smithi159.stdout: "upmap", 2024-09-19T07:37:18.515 INFO:teuthology.orchestra.run.smithi159.stdout: "upmap-read" 2024-09-19T07:37:18.515 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.515 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Balancer mode", 2024-09-19T07:37:18.515 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.515 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.515 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.516 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_ids": { 2024-09-19T07:37:18.516 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "pool_ids", 2024-09-19T07:37:18.516 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.516 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.516 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.516 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.516 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.516 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.516 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.516 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T07:37:18.516 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.516 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.517 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.517 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.517 INFO:teuthology.orchestra.run.smithi159.stdout: "sleep_interval": { 2024-09-19T07:37:18.517 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "sleep_interval", 2024-09-19T07:37:18.517 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.517 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.517 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.517 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "60", 2024-09-19T07:37:18.517 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.517 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.517 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.517 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T07:37:18.517 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.517 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.518 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.518 INFO:teuthology.orchestra.run.smithi159.stdout: "upmap_max_deviation": { 2024-09-19T07:37:18.518 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "upmap_max_deviation", 2024-09-19T07:37:18.518 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.518 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.518 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.518 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5", 2024-09-19T07:37:18.518 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "1", 2024-09-19T07:37:18.518 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.518 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.518 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T07:37:18.518 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T07:37:18.519 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.519 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.519 INFO:teuthology.orchestra.run.smithi159.stdout: "upmap_max_optimizations": { 2024-09-19T07:37:18.519 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "upmap_max_optimizations", 2024-09-19T07:37:18.519 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "uint", 2024-09-19T07:37:18.519 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.519 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.519 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:18.519 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.519 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.519 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.519 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T07:37:18.520 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.520 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.520 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.520 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.520 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.520 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "cephadm", 2024-09-19T07:37:18.520 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.520 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.520 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.520 INFO:teuthology.orchestra.run.smithi159.stdout: "agent_down_multiplier": { 2024-09-19T07:37:18.520 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "agent_down_multiplier", 2024-09-19T07:37:18.521 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.521 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.521 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.521 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "3.0", 2024-09-19T07:37:18.521 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.521 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.521 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.521 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T07:37:18.521 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.521 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.521 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.521 INFO:teuthology.orchestra.run.smithi159.stdout: "agent_refresh_rate": { 2024-09-19T07:37:18.522 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "agent_refresh_rate", 2024-09-19T07:37:18.522 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.522 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.522 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.522 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "20", 2024-09-19T07:37:18.522 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.522 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.522 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.522 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T07:37:18.522 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.522 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.522 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.522 INFO:teuthology.orchestra.run.smithi159.stdout: "agent_starting_port": { 2024-09-19T07:37:18.522 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "agent_starting_port", 2024-09-19T07:37:18.523 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.523 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.523 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.523 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "4721", 2024-09-19T07:37:18.523 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.523 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.523 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.523 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T07:37:18.523 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.523 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.523 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.523 INFO:teuthology.orchestra.run.smithi159.stdout: "allow_ptrace": { 2024-09-19T07:37:18.523 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "allow_ptrace", 2024-09-19T07:37:18.524 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.524 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.524 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.524 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.524 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.524 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.524 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.524 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T07:37:18.524 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T07:37:18.524 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.524 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.525 INFO:teuthology.orchestra.run.smithi159.stdout: "autotune_interval": { 2024-09-19T07:37:18.525 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "autotune_interval", 2024-09-19T07:37:18.525 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.525 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.525 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.525 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "600", 2024-09-19T07:37:18.525 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.525 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.525 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.525 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T07:37:18.525 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.525 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.525 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.526 INFO:teuthology.orchestra.run.smithi159.stdout: "autotune_memory_target_ratio": { 2024-09-19T07:37:18.526 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T07:37:18.526 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.526 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.526 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.526 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0.7", 2024-09-19T07:37:18.526 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.526 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.526 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.526 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T07:37:18.526 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.526 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.526 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.527 INFO:teuthology.orchestra.run.smithi159.stdout: "cephadm_log_destination": { 2024-09-19T07:37:18.527 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "cephadm_log_destination", 2024-09-19T07:37:18.527 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.527 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.527 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.527 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.527 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.527 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.527 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.527 INFO:teuthology.orchestra.run.smithi159.stdout: "file", 2024-09-19T07:37:18.527 INFO:teuthology.orchestra.run.smithi159.stdout: "file,syslog", 2024-09-19T07:37:18.527 INFO:teuthology.orchestra.run.smithi159.stdout: "syslog" 2024-09-19T07:37:18.527 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.527 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T07:37:18.528 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.528 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.528 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.528 INFO:teuthology.orchestra.run.smithi159.stdout: "cgroups_split": { 2024-09-19T07:37:18.528 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "cgroups_split", 2024-09-19T07:37:18.528 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.528 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.528 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.528 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.528 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.528 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.528 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.528 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T07:37:18.529 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.529 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.529 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.529 INFO:teuthology.orchestra.run.smithi159.stdout: "config_checks_enabled": { 2024-09-19T07:37:18.529 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "config_checks_enabled", 2024-09-19T07:37:18.529 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.529 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.529 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.529 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.529 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.529 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.529 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.530 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T07:37:18.530 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.530 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.530 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.530 INFO:teuthology.orchestra.run.smithi159.stdout: "config_dashboard": { 2024-09-19T07:37:18.530 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "config_dashboard", 2024-09-19T07:37:18.530 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.530 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.530 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.530 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.530 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.530 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.531 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.531 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T07:37:18.531 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.531 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.531 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.531 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_alertmanager": { 2024-09-19T07:37:18.531 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_alertmanager", 2024-09-19T07:37:18.531 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.531 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.531 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.531 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T07:37:18.531 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.531 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.532 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.532 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Prometheus container image", 2024-09-19T07:37:18.532 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.532 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.532 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.532 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.532 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_base": { 2024-09-19T07:37:18.532 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_base", 2024-09-19T07:37:18.532 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.533 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.533 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.533 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T07:37:18.533 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.533 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.533 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.533 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Container image name, without the tag", 2024-09-19T07:37:18.533 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.533 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.533 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.533 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_elasticsearch": { 2024-09-19T07:37:18.533 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_elasticsearch", 2024-09-19T07:37:18.533 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.534 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.534 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.534 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T07:37:18.534 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.534 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.534 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.534 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "elasticsearch container image", 2024-09-19T07:37:18.534 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.534 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.534 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.534 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.534 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_grafana": { 2024-09-19T07:37:18.534 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_grafana", 2024-09-19T07:37:18.535 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.535 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.535 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.535 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T07:37:18.535 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.535 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.535 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.535 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Prometheus container image", 2024-09-19T07:37:18.535 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.535 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.535 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.535 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_haproxy": { 2024-09-19T07:37:18.535 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_haproxy", 2024-09-19T07:37:18.536 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.536 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.536 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.536 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T07:37:18.536 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.536 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.536 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.536 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "HAproxy container image", 2024-09-19T07:37:18.536 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.536 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.536 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.536 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.536 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_jaeger_agent": { 2024-09-19T07:37:18.537 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_jaeger_agent", 2024-09-19T07:37:18.537 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.537 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.537 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.537 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T07:37:18.537 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.537 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.537 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.537 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Jaeger agent container image", 2024-09-19T07:37:18.537 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.537 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.537 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.538 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_jaeger_collector": { 2024-09-19T07:37:18.538 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_jaeger_collector", 2024-09-19T07:37:18.538 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.538 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.538 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.538 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T07:37:18.538 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.538 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.538 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.538 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Jaeger collector container image", 2024-09-19T07:37:18.538 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.538 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.538 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.539 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_jaeger_query": { 2024-09-19T07:37:18.539 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_jaeger_query", 2024-09-19T07:37:18.539 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.539 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.539 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.539 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T07:37:18.539 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.539 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.539 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.539 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Jaeger query container image", 2024-09-19T07:37:18.539 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.540 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.540 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.540 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_keepalived": { 2024-09-19T07:37:18.540 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_keepalived", 2024-09-19T07:37:18.540 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.540 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.540 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.540 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T07:37:18.540 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.540 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.540 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.540 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Keepalived container image", 2024-09-19T07:37:18.541 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.541 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.541 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.541 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_loki": { 2024-09-19T07:37:18.541 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_loki", 2024-09-19T07:37:18.541 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.541 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.541 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.541 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T07:37:18.541 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.541 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.541 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.541 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Loki container image", 2024-09-19T07:37:18.542 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.542 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.542 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.542 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_node_exporter": { 2024-09-19T07:37:18.542 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_node_exporter", 2024-09-19T07:37:18.542 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.542 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.542 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.542 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T07:37:18.542 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.542 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.542 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.543 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Prometheus container image", 2024-09-19T07:37:18.543 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.543 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.543 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.543 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_nvmeof": { 2024-09-19T07:37:18.543 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_nvmeof", 2024-09-19T07:37:18.543 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.543 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.543 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.543 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T07:37:18.543 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.543 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.544 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.544 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Nvme-of container image", 2024-09-19T07:37:18.544 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.544 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.544 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.544 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_prometheus": { 2024-09-19T07:37:18.544 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_prometheus", 2024-09-19T07:37:18.544 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.544 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.544 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.544 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T07:37:18.544 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.544 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.545 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.545 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Prometheus container image", 2024-09-19T07:37:18.545 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.545 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.545 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.545 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_promtail": { 2024-09-19T07:37:18.545 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_promtail", 2024-09-19T07:37:18.545 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.545 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.545 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.545 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T07:37:18.545 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.546 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.546 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.546 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Promtail container image", 2024-09-19T07:37:18.546 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.546 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.546 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.546 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_samba": { 2024-09-19T07:37:18.546 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_samba", 2024-09-19T07:37:18.546 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.546 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.546 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.546 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T07:37:18.546 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.547 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.547 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.547 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Samba/SMB container image", 2024-09-19T07:37:18.547 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.547 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.547 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.547 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_snmp_gateway": { 2024-09-19T07:37:18.547 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_snmp_gateway", 2024-09-19T07:37:18.547 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.547 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.547 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.547 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T07:37:18.548 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.548 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.548 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.548 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "SNMP Gateway container image", 2024-09-19T07:37:18.548 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.548 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.548 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.548 INFO:teuthology.orchestra.run.smithi159.stdout: "container_init": { 2024-09-19T07:37:18.548 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_init", 2024-09-19T07:37:18.548 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.548 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.549 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.549 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.549 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.549 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.549 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.549 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T07:37:18.549 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.549 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.549 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.549 INFO:teuthology.orchestra.run.smithi159.stdout: "daemon_cache_timeout": { 2024-09-19T07:37:18.549 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "daemon_cache_timeout", 2024-09-19T07:37:18.550 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.550 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.550 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.550 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "600", 2024-09-19T07:37:18.550 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.550 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.550 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.550 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T07:37:18.550 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.550 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.550 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.550 INFO:teuthology.orchestra.run.smithi159.stdout: "default_cephadm_command_timeout": { 2024-09-19T07:37:18.551 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T07:37:18.551 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.551 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.551 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.551 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "900", 2024-09-19T07:37:18.551 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.551 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.551 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.551 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T07:37:18.551 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.551 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.551 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.551 INFO:teuthology.orchestra.run.smithi159.stdout: "default_registry": { 2024-09-19T07:37:18.552 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "default_registry", 2024-09-19T07:37:18.552 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.552 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.552 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.552 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "docker.io", 2024-09-19T07:37:18.552 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.552 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.552 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.552 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T07:37:18.552 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.552 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.552 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.553 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.553 INFO:teuthology.orchestra.run.smithi159.stdout: "device_cache_timeout": { 2024-09-19T07:37:18.553 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "device_cache_timeout", 2024-09-19T07:37:18.553 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.553 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.553 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.553 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "1800", 2024-09-19T07:37:18.553 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.553 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.553 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.553 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "seconds to cache device inventory", 2024-09-19T07:37:18.553 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.554 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.554 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.554 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.554 INFO:teuthology.orchestra.run.smithi159.stdout: "device_enhanced_scan": { 2024-09-19T07:37:18.554 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "device_enhanced_scan", 2024-09-19T07:37:18.554 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.554 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.554 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.554 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.554 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.554 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.554 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.555 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T07:37:18.555 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.555 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.555 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.555 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.555 INFO:teuthology.orchestra.run.smithi159.stdout: "facts_cache_timeout": { 2024-09-19T07:37:18.555 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "facts_cache_timeout", 2024-09-19T07:37:18.555 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.555 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.555 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.555 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "60", 2024-09-19T07:37:18.555 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.555 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.556 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.556 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "seconds to cache host facts data", 2024-09-19T07:37:18.556 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.556 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.556 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.556 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.556 INFO:teuthology.orchestra.run.smithi159.stdout: "grafana_dashboards_path": { 2024-09-19T07:37:18.556 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "grafana_dashboards_path", 2024-09-19T07:37:18.556 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.556 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.556 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.556 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T07:37:18.556 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.557 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.557 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.557 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T07:37:18.557 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.557 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.557 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.557 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.557 INFO:teuthology.orchestra.run.smithi159.stdout: "host_check_interval": { 2024-09-19T07:37:18.557 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "host_check_interval", 2024-09-19T07:37:18.557 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.557 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.557 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.557 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "600", 2024-09-19T07:37:18.557 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.558 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.558 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.558 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how frequently to perform a host check", 2024-09-19T07:37:18.558 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.558 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.558 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.558 INFO:teuthology.orchestra.run.smithi159.stdout: "hw_monitoring": { 2024-09-19T07:37:18.558 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "hw_monitoring", 2024-09-19T07:37:18.558 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.558 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.558 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.558 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.559 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.559 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.559 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.559 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T07:37:18.559 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.559 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.559 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.559 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.559 INFO:teuthology.orchestra.run.smithi159.stdout: "inventory_list_all": { 2024-09-19T07:37:18.559 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "inventory_list_all", 2024-09-19T07:37:18.559 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.559 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.559 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.559 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.559 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.560 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.560 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.560 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T07:37:18.560 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.560 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.560 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.560 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.560 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.560 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.560 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.560 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.560 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.560 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.560 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.560 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.561 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.561 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.561 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.561 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.561 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.561 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.561 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.561 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.561 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.561 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.561 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.561 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.561 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.561 INFO:teuthology.orchestra.run.smithi159.stdout: "log_refresh_metadata": { 2024-09-19T07:37:18.562 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_refresh_metadata", 2024-09-19T07:37:18.562 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.562 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.562 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.562 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.562 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.562 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.562 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.562 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T07:37:18.562 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.562 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.562 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.562 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.562 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.562 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.563 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.563 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.563 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.563 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.563 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.563 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.563 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.563 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T07:37:18.563 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.563 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.563 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.563 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.563 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.563 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.563 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.564 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.564 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.564 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.564 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.564 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.564 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.564 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.564 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.564 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.564 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.564 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.564 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.565 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.565 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.565 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.565 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.565 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.565 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.565 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.565 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.565 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.565 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.565 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.566 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.566 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.566 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.566 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.566 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.566 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.566 INFO:teuthology.orchestra.run.smithi159.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T07:37:18.566 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T07:37:18.566 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.566 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.566 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.566 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.566 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.567 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.567 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.567 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T07:37:18.567 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.567 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.567 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.567 INFO:teuthology.orchestra.run.smithi159.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T07:37:18.567 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T07:37:18.567 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.567 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.567 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.567 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "*", 2024-09-19T07:37:18.567 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.567 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.567 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.568 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T07:37:18.568 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.568 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.568 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.568 INFO:teuthology.orchestra.run.smithi159.stdout: "max_count_per_host": { 2024-09-19T07:37:18.568 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "max_count_per_host", 2024-09-19T07:37:18.568 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.568 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.568 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.568 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:18.568 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.568 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.568 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.569 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "max number of daemons per service per host", 2024-09-19T07:37:18.569 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.569 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.569 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.569 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.569 INFO:teuthology.orchestra.run.smithi159.stdout: "max_osd_draining_count": { 2024-09-19T07:37:18.569 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "max_osd_draining_count", 2024-09-19T07:37:18.569 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.569 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.569 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.570 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:18.570 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.570 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.570 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.570 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T07:37:18.570 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.570 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.570 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.570 INFO:teuthology.orchestra.run.smithi159.stdout: "migration_current": { 2024-09-19T07:37:18.570 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "migration_current", 2024-09-19T07:37:18.570 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.570 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.570 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.571 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.571 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.571 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.571 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.571 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "internal - do not modify", 2024-09-19T07:37:18.571 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.571 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.571 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.571 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.571 INFO:teuthology.orchestra.run.smithi159.stdout: "mode": { 2024-09-19T07:37:18.571 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mode", 2024-09-19T07:37:18.572 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.572 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.572 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.572 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "root", 2024-09-19T07:37:18.572 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.572 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.572 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.572 INFO:teuthology.orchestra.run.smithi159.stdout: "cephadm-package", 2024-09-19T07:37:18.572 INFO:teuthology.orchestra.run.smithi159.stdout: "root" 2024-09-19T07:37:18.572 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.572 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T07:37:18.572 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.572 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.573 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.573 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.573 INFO:teuthology.orchestra.run.smithi159.stdout: "oob_default_addr": { 2024-09-19T07:37:18.573 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "oob_default_addr", 2024-09-19T07:37:18.573 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.573 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.573 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.573 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "169.254.1.1", 2024-09-19T07:37:18.573 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.573 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.573 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.573 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T07:37:18.573 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.573 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.574 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.574 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.574 INFO:teuthology.orchestra.run.smithi159.stdout: "prometheus_alerts_path": { 2024-09-19T07:37:18.574 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "prometheus_alerts_path", 2024-09-19T07:37:18.574 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.574 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.574 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.574 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T07:37:18.574 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.574 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.574 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.574 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T07:37:18.574 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.574 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.575 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.575 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.575 INFO:teuthology.orchestra.run.smithi159.stdout: "registry_insecure": { 2024-09-19T07:37:18.575 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "registry_insecure", 2024-09-19T07:37:18.575 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.575 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.575 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.575 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.575 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.575 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.575 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.575 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T07:37:18.575 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.576 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.576 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.576 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.576 INFO:teuthology.orchestra.run.smithi159.stdout: "registry_password": { 2024-09-19T07:37:18.576 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "registry_password", 2024-09-19T07:37:18.576 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.576 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.576 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.576 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.576 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.576 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.576 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.576 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T07:37:18.576 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.577 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.577 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.577 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.577 INFO:teuthology.orchestra.run.smithi159.stdout: "registry_url": { 2024-09-19T07:37:18.577 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "registry_url", 2024-09-19T07:37:18.577 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.577 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.577 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.577 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.577 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.577 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.577 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.577 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T07:37:18.578 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.578 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.578 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.578 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.578 INFO:teuthology.orchestra.run.smithi159.stdout: "registry_username": { 2024-09-19T07:37:18.578 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "registry_username", 2024-09-19T07:37:18.578 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.578 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.578 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.578 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.578 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.578 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.578 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.578 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T07:37:18.579 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.579 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.579 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.579 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.579 INFO:teuthology.orchestra.run.smithi159.stdout: "secure_monitoring_stack": { 2024-09-19T07:37:18.579 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "secure_monitoring_stack", 2024-09-19T07:37:18.579 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.579 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.579 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.579 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.579 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.579 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.579 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.580 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T07:37:18.580 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.580 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.580 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.580 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.580 INFO:teuthology.orchestra.run.smithi159.stdout: "service_discovery_port": { 2024-09-19T07:37:18.580 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "service_discovery_port", 2024-09-19T07:37:18.580 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.580 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.580 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.580 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "8765", 2024-09-19T07:37:18.580 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.580 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.581 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.581 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "cephadm service discovery port", 2024-09-19T07:37:18.581 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.581 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.581 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.581 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.581 INFO:teuthology.orchestra.run.smithi159.stdout: "ssh_config_file": { 2024-09-19T07:37:18.581 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ssh_config_file", 2024-09-19T07:37:18.581 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.581 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.581 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.581 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.581 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.582 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.582 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.582 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T07:37:18.582 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.582 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.582 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.582 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.582 INFO:teuthology.orchestra.run.smithi159.stdout: "use_agent": { 2024-09-19T07:37:18.582 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "use_agent", 2024-09-19T07:37:18.582 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.582 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.582 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.582 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.582 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.582 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.583 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.583 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T07:37:18.583 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.583 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.583 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.583 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.583 INFO:teuthology.orchestra.run.smithi159.stdout: "use_repo_digest": { 2024-09-19T07:37:18.583 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "use_repo_digest", 2024-09-19T07:37:18.583 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.583 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.583 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.583 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.583 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.583 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.583 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.584 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T07:37:18.584 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.584 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.584 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.584 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.584 INFO:teuthology.orchestra.run.smithi159.stdout: "warn_on_failed_host_check": { 2024-09-19T07:37:18.584 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "warn_on_failed_host_check", 2024-09-19T07:37:18.584 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.584 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.584 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.584 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.584 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.584 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.584 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.585 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T07:37:18.585 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.585 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.585 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.585 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.585 INFO:teuthology.orchestra.run.smithi159.stdout: "warn_on_stray_daemons": { 2024-09-19T07:37:18.585 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "warn_on_stray_daemons", 2024-09-19T07:37:18.585 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.585 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.585 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.585 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.585 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.585 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.586 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.586 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T07:37:18.586 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.586 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.586 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.586 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.586 INFO:teuthology.orchestra.run.smithi159.stdout: "warn_on_stray_hosts": { 2024-09-19T07:37:18.586 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "warn_on_stray_hosts", 2024-09-19T07:37:18.586 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.586 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.586 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.586 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.586 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.586 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.587 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.587 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T07:37:18.587 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.587 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.587 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.587 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.587 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.587 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.587 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.587 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "crash", 2024-09-19T07:37:18.587 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.587 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.587 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.588 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.588 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.588 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.588 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.588 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.588 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.588 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.588 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.588 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.588 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.588 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.588 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.588 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.588 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.589 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.589 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.589 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.589 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.589 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.589 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.589 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.589 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.589 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.589 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.589 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.589 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.589 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.589 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.590 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.590 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.590 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.590 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.590 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.590 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.590 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.590 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.590 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.590 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.590 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.590 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.590 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.590 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.591 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.591 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.591 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.591 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.591 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.591 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.591 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.591 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.591 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.591 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.591 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.591 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.591 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.591 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.592 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.592 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.592 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.592 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.592 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.592 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.592 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.592 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.592 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.592 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.592 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.592 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.592 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.592 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.593 INFO:teuthology.orchestra.run.smithi159.stdout: "retain_interval": { 2024-09-19T07:37:18.593 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "retain_interval", 2024-09-19T07:37:18.593 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.593 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.593 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.593 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "31536000", 2024-09-19T07:37:18.593 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.593 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.593 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.593 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T07:37:18.593 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.593 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.594 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.594 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.594 INFO:teuthology.orchestra.run.smithi159.stdout: "warn_recent_interval": { 2024-09-19T07:37:18.594 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "warn_recent_interval", 2024-09-19T07:37:18.594 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.594 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.594 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.594 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "1209600", 2024-09-19T07:37:18.594 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.594 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.594 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.596 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T07:37:18.596 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.596 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.596 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.596 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.596 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.596 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.596 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.597 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "dashboard", 2024-09-19T07:37:18.597 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.597 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.597 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.597 INFO:teuthology.orchestra.run.smithi159.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T07:37:18.597 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T07:37:18.597 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.597 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.597 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.597 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:18.597 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.597 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.598 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.598 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.598 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.598 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.598 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.598 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.598 INFO:teuthology.orchestra.run.smithi159.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T07:37:18.598 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T07:37:18.598 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.598 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.598 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.598 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.599 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.599 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.599 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.599 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.599 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.599 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.599 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.599 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.599 INFO:teuthology.orchestra.run.smithi159.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T07:37:18.599 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T07:37:18.599 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.599 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.599 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.600 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.600 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.600 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.600 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.600 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.600 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.600 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.600 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.600 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.600 INFO:teuthology.orchestra.run.smithi159.stdout: "AUDIT_API_ENABLED": { 2024-09-19T07:37:18.600 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T07:37:18.600 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.600 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.601 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.601 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.601 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.601 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.601 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.601 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.601 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.601 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.601 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.601 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.601 INFO:teuthology.orchestra.run.smithi159.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T07:37:18.601 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T07:37:18.602 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.602 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.602 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.602 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.602 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.602 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.602 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.602 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.602 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.602 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.602 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.602 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.602 INFO:teuthology.orchestra.run.smithi159.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T07:37:18.603 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T07:37:18.603 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.603 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.603 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.603 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.603 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.603 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.603 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.603 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.603 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.603 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.603 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.603 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.604 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T07:37:18.604 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T07:37:18.604 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.604 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.604 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.604 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.604 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.604 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.604 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.604 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.604 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.604 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.604 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.605 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.605 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T07:37:18.605 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T07:37:18.605 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.605 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.605 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.605 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.605 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.605 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.605 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.605 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.605 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.606 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.606 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.606 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.606 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T07:37:18.606 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T07:37:18.606 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.606 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.606 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.606 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.606 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.606 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.606 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.606 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.607 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.607 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.607 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.607 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.607 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T07:37:18.607 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T07:37:18.607 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.607 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.607 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.607 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.607 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.607 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.607 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.607 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.608 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.608 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.608 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.608 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.608 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T07:37:18.608 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T07:37:18.608 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.608 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.608 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.608 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.608 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.608 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.608 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.609 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.609 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.609 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.609 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.609 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.609 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T07:37:18.609 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T07:37:18.609 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.609 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.609 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.609 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.609 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.609 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.610 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.610 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.610 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.610 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.610 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.610 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.610 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T07:37:18.610 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T07:37:18.610 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.610 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.610 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.610 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.610 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.610 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.611 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.611 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.611 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.611 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.611 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.611 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.611 INFO:teuthology.orchestra.run.smithi159.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T07:37:18.611 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T07:37:18.611 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.611 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.611 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.611 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.611 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.612 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.612 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.612 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.612 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.612 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.612 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.612 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.612 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T07:37:18.612 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T07:37:18.612 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.612 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.612 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.612 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "admin", 2024-09-19T07:37:18.613 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.613 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.613 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.613 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.613 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.613 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.613 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.613 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.613 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T07:37:18.613 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T07:37:18.613 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.613 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.613 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.614 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.614 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.614 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.614 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.614 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.614 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.614 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.614 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.614 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.614 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_API_URL": { 2024-09-19T07:37:18.614 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_API_URL", 2024-09-19T07:37:18.614 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.614 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.614 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.615 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.615 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.615 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.615 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.615 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.615 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.615 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.615 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.615 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.615 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T07:37:18.615 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T07:37:18.615 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.616 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.616 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.616 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "admin", 2024-09-19T07:37:18.616 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.616 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.616 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.616 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.616 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.616 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.616 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.616 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.616 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T07:37:18.616 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T07:37:18.616 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.617 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.617 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.617 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.617 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.617 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.617 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.617 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.617 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.617 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.617 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.617 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.617 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T07:37:18.617 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T07:37:18.617 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.618 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.618 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.618 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.618 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.618 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.618 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.618 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.618 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.618 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.618 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.618 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.618 INFO:teuthology.orchestra.run.smithi159.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T07:37:18.618 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T07:37:18.618 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.618 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.619 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.619 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.619 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.619 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.619 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.619 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.619 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.619 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.619 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.619 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.619 INFO:teuthology.orchestra.run.smithi159.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T07:37:18.619 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T07:37:18.619 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.619 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.619 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.619 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.620 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.620 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.620 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.620 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.620 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.620 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.620 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.620 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.620 INFO:teuthology.orchestra.run.smithi159.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T07:37:18.620 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T07:37:18.620 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.620 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.620 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.620 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.621 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.621 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.621 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.621 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.621 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.621 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.621 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.621 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.621 INFO:teuthology.orchestra.run.smithi159.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T07:37:18.621 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T07:37:18.621 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.621 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.621 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.621 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.622 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.622 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.622 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.622 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.622 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.622 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.622 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.622 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.622 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T07:37:18.622 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T07:37:18.622 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.622 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.622 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.622 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.623 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.623 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.623 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.623 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.623 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.623 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.623 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.623 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.623 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T07:37:18.623 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T07:37:18.623 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.623 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.623 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.624 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.624 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.624 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.624 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.624 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.624 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.624 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.624 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.624 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.624 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T07:37:18.624 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T07:37:18.624 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.624 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.624 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.625 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.625 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.625 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.625 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.625 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.625 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.625 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.625 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.625 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.625 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T07:37:18.625 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T07:37:18.625 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.625 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.625 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.625 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.626 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.626 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.626 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.626 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.626 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.626 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.626 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.626 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.626 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T07:37:18.626 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T07:37:18.626 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.626 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.626 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.626 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.627 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.627 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.627 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.627 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.627 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.627 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.627 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.627 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.627 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T07:37:18.627 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T07:37:18.627 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.627 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.627 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.627 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.628 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.628 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.628 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.628 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.628 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.628 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.628 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.628 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.628 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T07:37:18.628 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T07:37:18.628 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.628 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.628 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.628 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.628 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.628 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.629 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.629 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.629 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.629 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.629 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.629 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.629 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T07:37:18.629 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T07:37:18.629 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.629 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.629 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.629 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.629 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.629 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.630 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.630 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.630 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.630 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.630 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.630 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T07:37:18.630 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T07:37:18.630 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.630 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.630 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.630 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T07:37:18.630 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.630 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.631 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.631 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.631 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.631 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.631 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.631 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.631 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T07:37:18.631 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T07:37:18.631 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.631 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.631 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.631 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:18.631 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.631 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.632 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.632 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.632 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.632 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.632 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.632 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.632 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T07:37:18.632 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T07:37:18.632 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.633 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.633 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.633 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "8", 2024-09-19T07:37:18.633 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.633 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.633 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.633 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.633 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.633 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.633 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.633 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.633 INFO:teuthology.orchestra.run.smithi159.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T07:37:18.633 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T07:37:18.633 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.633 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.634 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.634 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "45", 2024-09-19T07:37:18.634 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.634 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.634 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.634 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.634 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.634 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.634 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.634 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.634 INFO:teuthology.orchestra.run.smithi159.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T07:37:18.634 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T07:37:18.634 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.635 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.635 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.635 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.635 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.635 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.635 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.635 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.635 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.635 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.635 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.635 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.635 INFO:teuthology.orchestra.run.smithi159.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T07:37:18.635 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T07:37:18.635 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.636 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.636 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.636 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "admin", 2024-09-19T07:37:18.636 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.636 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.636 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.636 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.636 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.636 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.636 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.636 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.636 INFO:teuthology.orchestra.run.smithi159.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T07:37:18.636 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T07:37:18.636 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.637 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.637 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.637 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.637 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.637 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.637 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.637 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.637 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.637 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.637 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.637 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.637 INFO:teuthology.orchestra.run.smithi159.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T07:37:18.637 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T07:37:18.638 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.638 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.638 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.638 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.638 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.638 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.638 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.638 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.638 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.638 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.638 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.638 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.638 INFO:teuthology.orchestra.run.smithi159.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T07:37:18.639 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T07:37:18.639 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.639 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.639 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.639 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0", 2024-09-19T07:37:18.639 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.639 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.639 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.639 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.639 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.639 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.639 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.639 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.639 INFO:teuthology.orchestra.run.smithi159.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T07:37:18.640 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T07:37:18.640 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.640 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.640 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.640 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:18.640 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.640 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.640 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.640 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.640 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.640 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.640 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.640 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.641 INFO:teuthology.orchestra.run.smithi159.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T07:37:18.641 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T07:37:18.641 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.641 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.641 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.641 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5", 2024-09-19T07:37:18.641 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.641 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.641 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.641 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.641 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.641 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.641 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.641 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.642 INFO:teuthology.orchestra.run.smithi159.stdout: "cross_origin_url": { 2024-09-19T07:37:18.642 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "cross_origin_url", 2024-09-19T07:37:18.642 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.642 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.642 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.642 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.642 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.642 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.642 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.642 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.642 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.642 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.642 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.642 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.642 INFO:teuthology.orchestra.run.smithi159.stdout: "crt_file": { 2024-09-19T07:37:18.643 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "crt_file", 2024-09-19T07:37:18.643 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.643 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.643 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.643 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.643 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.643 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.643 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.643 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.643 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.643 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.643 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.643 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.643 INFO:teuthology.orchestra.run.smithi159.stdout: "debug": { 2024-09-19T07:37:18.643 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "debug", 2024-09-19T07:37:18.644 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.644 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.644 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.644 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.644 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.644 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.644 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.644 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Enable/disable debug options", 2024-09-19T07:37:18.644 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.644 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.644 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.644 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.644 INFO:teuthology.orchestra.run.smithi159.stdout: "jwt_token_ttl": { 2024-09-19T07:37:18.645 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "jwt_token_ttl", 2024-09-19T07:37:18.645 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.645 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.645 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.645 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "28800", 2024-09-19T07:37:18.645 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.645 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.645 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.645 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.645 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.645 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.645 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.645 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.646 INFO:teuthology.orchestra.run.smithi159.stdout: "key_file": { 2024-09-19T07:37:18.646 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "key_file", 2024-09-19T07:37:18.646 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.646 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.646 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.646 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.646 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.646 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.646 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.646 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.646 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.646 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.646 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.646 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.647 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.647 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.647 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.647 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.647 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.647 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.647 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.647 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.647 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.647 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.647 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.647 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.647 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.648 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.648 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.648 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.648 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.648 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.648 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.648 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.648 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.648 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.648 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.648 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.648 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.648 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.649 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.649 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.649 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.649 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.649 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.649 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.649 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.649 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.649 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.649 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.649 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.649 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.650 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.650 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.650 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.650 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.650 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.650 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.650 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.650 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.650 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.650 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.650 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.650 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.650 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.650 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.651 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.651 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.651 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.651 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.651 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.651 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.651 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.651 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.651 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.651 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.651 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.651 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.651 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.651 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.652 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.652 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.652 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.652 INFO:teuthology.orchestra.run.smithi159.stdout: "motd": { 2024-09-19T07:37:18.652 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "motd", 2024-09-19T07:37:18.652 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.652 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.652 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.652 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.652 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.652 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.652 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.652 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "The message of the day", 2024-09-19T07:37:18.653 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.653 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.653 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.653 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.653 INFO:teuthology.orchestra.run.smithi159.stdout: "redirect_resolve_ip_addr": { 2024-09-19T07:37:18.653 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T07:37:18.653 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.653 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.653 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.653 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.653 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.653 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.653 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.654 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.654 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.654 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.654 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.654 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.654 INFO:teuthology.orchestra.run.smithi159.stdout: "server_addr": { 2024-09-19T07:37:18.654 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_addr", 2024-09-19T07:37:18.654 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.654 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.654 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.654 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "::", 2024-09-19T07:37:18.654 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.654 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.654 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.655 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.655 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.655 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.655 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.655 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.655 INFO:teuthology.orchestra.run.smithi159.stdout: "server_port": { 2024-09-19T07:37:18.655 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_port", 2024-09-19T07:37:18.655 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.655 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.655 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.655 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "8080", 2024-09-19T07:37:18.655 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.655 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.655 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.655 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.656 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.656 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.656 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.656 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.656 INFO:teuthology.orchestra.run.smithi159.stdout: "ssl": { 2024-09-19T07:37:18.656 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ssl", 2024-09-19T07:37:18.656 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.656 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.656 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.656 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.656 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.656 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.656 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.656 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.657 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.657 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.657 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.657 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.657 INFO:teuthology.orchestra.run.smithi159.stdout: "ssl_server_port": { 2024-09-19T07:37:18.657 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ssl_server_port", 2024-09-19T07:37:18.657 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.657 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.657 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.657 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "8443", 2024-09-19T07:37:18.657 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.657 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.657 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.657 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.658 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.658 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.658 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.658 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.658 INFO:teuthology.orchestra.run.smithi159.stdout: "standby_behaviour": { 2024-09-19T07:37:18.658 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "standby_behaviour", 2024-09-19T07:37:18.658 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.658 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.658 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.658 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "redirect", 2024-09-19T07:37:18.658 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.658 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.658 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.658 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.659 INFO:teuthology.orchestra.run.smithi159.stdout: "redirect" 2024-09-19T07:37:18.659 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.659 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.659 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.659 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.659 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.659 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.659 INFO:teuthology.orchestra.run.smithi159.stdout: "standby_error_status_code": { 2024-09-19T07:37:18.659 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "standby_error_status_code", 2024-09-19T07:37:18.659 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.659 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.659 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.659 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "500", 2024-09-19T07:37:18.659 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "400", 2024-09-19T07:37:18.659 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "599", 2024-09-19T07:37:18.660 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.660 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.660 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.660 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.660 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.660 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.660 INFO:teuthology.orchestra.run.smithi159.stdout: "url_prefix": { 2024-09-19T07:37:18.660 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "url_prefix", 2024-09-19T07:37:18.660 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.660 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.660 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.660 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.660 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.661 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.661 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.661 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.661 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.661 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.661 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.661 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.661 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.661 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.661 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.661 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "devicehealth", 2024-09-19T07:37:18.661 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.661 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.661 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.662 INFO:teuthology.orchestra.run.smithi159.stdout: "enable_monitoring": { 2024-09-19T07:37:18.662 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "enable_monitoring", 2024-09-19T07:37:18.662 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.662 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.662 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.662 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.662 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.662 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.662 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.662 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "monitor device health metrics", 2024-09-19T07:37:18.662 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.662 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.662 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.662 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.662 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.663 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.663 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.663 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.663 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.663 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.663 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.663 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.663 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.663 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.663 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.663 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.663 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.663 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.663 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.664 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.664 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.664 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.664 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.664 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.664 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.664 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.664 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.664 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.664 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.664 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.664 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.664 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.665 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.665 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.665 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.665 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.665 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.665 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.665 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.665 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.665 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.665 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.665 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.665 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.665 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.665 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.666 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.666 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.666 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.666 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.666 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.666 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.666 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.666 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.666 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.666 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.666 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.666 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.666 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.666 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.667 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.667 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.667 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.667 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.667 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.667 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.667 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.667 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.667 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.667 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.667 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.667 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.667 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.668 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.668 INFO:teuthology.orchestra.run.smithi159.stdout: "mark_out_threshold": { 2024-09-19T07:37:18.668 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mark_out_threshold", 2024-09-19T07:37:18.668 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.668 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.668 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.668 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "2419200", 2024-09-19T07:37:18.668 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.668 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.668 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.668 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T07:37:18.668 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.668 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.668 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.669 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_name": { 2024-09-19T07:37:18.669 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "pool_name", 2024-09-19T07:37:18.669 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.669 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.669 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.669 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "device_health_metrics", 2024-09-19T07:37:18.669 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.669 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.669 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.669 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T07:37:18.669 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.669 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.670 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.670 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.670 INFO:teuthology.orchestra.run.smithi159.stdout: "retention_period": { 2024-09-19T07:37:18.670 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "retention_period", 2024-09-19T07:37:18.670 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.670 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.670 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.670 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "15552000", 2024-09-19T07:37:18.670 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.670 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.670 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.670 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how long to retain device health metrics", 2024-09-19T07:37:18.670 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.670 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.670 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.671 INFO:teuthology.orchestra.run.smithi159.stdout: "scrape_frequency": { 2024-09-19T07:37:18.671 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "scrape_frequency", 2024-09-19T07:37:18.671 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.671 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.671 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.671 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "86400", 2024-09-19T07:37:18.671 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.671 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.671 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.671 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T07:37:18.671 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.672 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.672 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.672 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.672 INFO:teuthology.orchestra.run.smithi159.stdout: "self_heal": { 2024-09-19T07:37:18.672 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "self_heal", 2024-09-19T07:37:18.672 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.672 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.672 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.672 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.672 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.672 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.672 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.672 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T07:37:18.672 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.673 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.673 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.673 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.673 INFO:teuthology.orchestra.run.smithi159.stdout: "sleep_interval": { 2024-09-19T07:37:18.673 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "sleep_interval", 2024-09-19T07:37:18.673 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.673 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.673 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.673 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "600", 2024-09-19T07:37:18.673 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.673 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.673 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.673 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T07:37:18.673 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.674 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.674 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.674 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.674 INFO:teuthology.orchestra.run.smithi159.stdout: "warn_threshold": { 2024-09-19T07:37:18.674 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "warn_threshold", 2024-09-19T07:37:18.674 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.674 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.674 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.674 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "7257600", 2024-09-19T07:37:18.674 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.674 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.674 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.675 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T07:37:18.675 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.675 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.675 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.675 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.675 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.675 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.675 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.675 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "diskprediction_local", 2024-09-19T07:37:18.675 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.675 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.675 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.675 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.676 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.676 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.676 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.676 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.676 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.676 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.676 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.676 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.676 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.676 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.676 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.676 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.676 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.676 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.677 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.677 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.677 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.677 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.677 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.677 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.677 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.677 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.677 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.677 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.677 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.677 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.677 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.678 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.678 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.678 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.678 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.678 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.678 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.678 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.678 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.678 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.678 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.678 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.678 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.678 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.679 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.679 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.679 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.679 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.679 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.679 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.679 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.679 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.679 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.679 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.679 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.679 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.679 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.680 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.680 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.680 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.680 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.680 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.680 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.680 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.680 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.680 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.680 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.680 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.680 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.680 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.681 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.681 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.681 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.681 INFO:teuthology.orchestra.run.smithi159.stdout: "predict_interval": { 2024-09-19T07:37:18.681 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "predict_interval", 2024-09-19T07:37:18.681 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.681 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.681 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.681 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "86400", 2024-09-19T07:37:18.681 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.681 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.681 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.681 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.681 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.682 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.682 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.682 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.682 INFO:teuthology.orchestra.run.smithi159.stdout: "predictor_model": { 2024-09-19T07:37:18.682 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "predictor_model", 2024-09-19T07:37:18.682 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.682 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.682 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.682 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "prophetstor", 2024-09-19T07:37:18.682 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.682 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.682 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.682 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.682 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.683 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.683 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.683 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.683 INFO:teuthology.orchestra.run.smithi159.stdout: "sleep_interval": { 2024-09-19T07:37:18.683 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "sleep_interval", 2024-09-19T07:37:18.683 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.683 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.683 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.683 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "600", 2024-09-19T07:37:18.683 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.683 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.683 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.683 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.683 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.683 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.684 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.684 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.684 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.684 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.684 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.684 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "influx", 2024-09-19T07:37:18.684 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": false, 2024-09-19T07:37:18.684 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "influxdb python module not found", 2024-09-19T07:37:18.684 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.684 INFO:teuthology.orchestra.run.smithi159.stdout: "batch_size": { 2024-09-19T07:37:18.684 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "batch_size", 2024-09-19T07:37:18.684 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.684 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.684 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.685 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5000", 2024-09-19T07:37:18.685 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.685 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.685 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.685 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T07:37:18.685 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.685 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.685 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.685 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.685 INFO:teuthology.orchestra.run.smithi159.stdout: "database": { 2024-09-19T07:37:18.685 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "database", 2024-09-19T07:37:18.685 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.685 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.686 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.686 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "ceph", 2024-09-19T07:37:18.686 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.686 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.686 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.686 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T07:37:18.686 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.686 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.686 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.686 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.686 INFO:teuthology.orchestra.run.smithi159.stdout: "hostname": { 2024-09-19T07:37:18.686 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "hostname", 2024-09-19T07:37:18.686 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.687 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.687 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.687 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.687 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.687 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.687 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.687 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "InfluxDB server hostname", 2024-09-19T07:37:18.687 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.687 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.687 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.687 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.687 INFO:teuthology.orchestra.run.smithi159.stdout: "interval": { 2024-09-19T07:37:18.687 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "interval", 2024-09-19T07:37:18.687 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.688 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.688 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.688 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "30", 2024-09-19T07:37:18.688 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "5", 2024-09-19T07:37:18.688 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.688 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.688 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T07:37:18.688 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.688 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.688 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.688 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.688 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.688 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.688 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.689 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.689 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.689 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.689 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.689 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.689 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.689 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.689 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.689 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.689 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.689 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.689 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.690 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.690 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.690 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.690 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.690 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.690 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.690 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.690 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.690 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.690 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.690 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.690 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.690 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.690 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.690 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.691 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.691 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.691 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.691 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.691 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.691 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.691 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.691 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.691 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.691 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.691 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.691 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.691 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.691 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.692 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.692 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.692 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.692 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.692 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.692 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.692 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.692 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.692 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.692 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.692 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.692 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.693 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.693 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.693 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.693 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.693 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.693 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.693 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.693 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.693 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.693 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.693 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.693 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.693 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.694 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.694 INFO:teuthology.orchestra.run.smithi159.stdout: "password": { 2024-09-19T07:37:18.694 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "password", 2024-09-19T07:37:18.694 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.694 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.694 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.694 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.694 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.694 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.694 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.694 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "password of InfluxDB server user", 2024-09-19T07:37:18.694 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.694 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.695 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.695 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.695 INFO:teuthology.orchestra.run.smithi159.stdout: "port": { 2024-09-19T07:37:18.695 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "port", 2024-09-19T07:37:18.695 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.695 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.695 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.695 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "8086", 2024-09-19T07:37:18.695 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.695 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.695 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.695 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "InfluxDB server port", 2024-09-19T07:37:18.695 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.695 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.696 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.696 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.696 INFO:teuthology.orchestra.run.smithi159.stdout: "ssl": { 2024-09-19T07:37:18.696 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ssl", 2024-09-19T07:37:18.696 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.696 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.696 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.696 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "false", 2024-09-19T07:37:18.696 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.696 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.696 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.696 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:37:18.696 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.696 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.697 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.697 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.697 INFO:teuthology.orchestra.run.smithi159.stdout: "threads": { 2024-09-19T07:37:18.697 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "threads", 2024-09-19T07:37:18.697 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.697 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.697 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.697 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5", 2024-09-19T07:37:18.697 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "1", 2024-09-19T07:37:18.697 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "32", 2024-09-19T07:37:18.697 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.697 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T07:37:18.697 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.697 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.698 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.698 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.698 INFO:teuthology.orchestra.run.smithi159.stdout: "username": { 2024-09-19T07:37:18.698 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "username", 2024-09-19T07:37:18.698 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.698 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.698 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.698 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.698 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.698 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.698 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.698 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "username of InfluxDB server user", 2024-09-19T07:37:18.698 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.698 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.699 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.699 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.699 INFO:teuthology.orchestra.run.smithi159.stdout: "verify_ssl": { 2024-09-19T07:37:18.699 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "verify_ssl", 2024-09-19T07:37:18.699 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.699 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.699 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.699 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "true", 2024-09-19T07:37:18.699 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.699 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.699 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.699 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:37:18.699 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.700 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.700 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.700 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.700 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.700 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.700 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.700 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "insights", 2024-09-19T07:37:18.700 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.700 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.700 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.700 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.700 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.700 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.700 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.701 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.701 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.701 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.701 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.701 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.701 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.701 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.701 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.701 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.701 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.701 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.701 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.701 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.702 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.702 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.702 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.702 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.702 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.702 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.702 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.702 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.702 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.702 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.702 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.702 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.702 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.702 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.703 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.703 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.703 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.703 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.703 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.703 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.703 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.703 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.703 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.703 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.703 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.703 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.703 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.703 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.704 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.704 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.704 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.704 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.704 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.704 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.704 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.704 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.704 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.704 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.704 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.704 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.704 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.704 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.705 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.705 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.705 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.705 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.705 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.705 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.705 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.705 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.705 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.705 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.705 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.705 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.705 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.705 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.706 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "iostat", 2024-09-19T07:37:18.706 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.706 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.706 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.706 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.706 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.706 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.706 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.706 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.706 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.706 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.706 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.706 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.706 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.707 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.707 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.707 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.707 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.707 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.707 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.707 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.707 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.707 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.707 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.707 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.707 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.707 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.708 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.708 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.708 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.708 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.708 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.708 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.708 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.708 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.708 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.708 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.708 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.708 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.708 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.708 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.708 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.709 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.709 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.709 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.709 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.709 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.709 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.709 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.709 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.709 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.709 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.709 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.709 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.709 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.709 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.710 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.710 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.710 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.710 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.710 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.710 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.710 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.710 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.710 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.710 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.710 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.710 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.710 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.710 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.710 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.711 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.711 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.711 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.711 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.711 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "k8sevents", 2024-09-19T07:37:18.711 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.711 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.711 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.711 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_event_retention_days": { 2024-09-19T07:37:18.711 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ceph_event_retention_days", 2024-09-19T07:37:18.711 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.711 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.711 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.712 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "7", 2024-09-19T07:37:18.712 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.712 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.712 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.712 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T07:37:18.712 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.712 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.712 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.712 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.712 INFO:teuthology.orchestra.run.smithi159.stdout: "config_check_secs": { 2024-09-19T07:37:18.712 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "config_check_secs", 2024-09-19T07:37:18.712 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.712 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.713 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.713 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:18.713 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "10", 2024-09-19T07:37:18.713 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.713 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.713 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T07:37:18.713 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.713 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.713 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.713 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.713 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.713 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.713 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.714 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.714 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.714 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.714 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.714 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.714 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.714 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.714 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.714 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.714 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.714 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.714 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.714 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.715 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.715 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.715 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.715 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.715 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.715 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.715 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.715 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.715 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.715 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.715 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.715 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.715 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.715 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.716 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.716 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.716 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.716 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.716 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.716 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.716 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.716 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.716 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.716 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.716 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.716 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.716 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.717 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.717 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.717 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.717 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.717 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.717 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.717 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.717 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.717 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.717 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.717 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.717 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.717 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.718 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.718 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.718 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.718 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.718 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.718 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.718 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.718 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.718 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.718 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.718 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.718 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.718 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.719 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.719 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.719 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.719 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.719 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "localpool", 2024-09-19T07:37:18.719 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.719 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.719 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.719 INFO:teuthology.orchestra.run.smithi159.stdout: "failure_domain": { 2024-09-19T07:37:18.719 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "failure_domain", 2024-09-19T07:37:18.719 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.719 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.719 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.720 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "host", 2024-09-19T07:37:18.720 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.720 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.720 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.720 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "failure domain for any created local pool", 2024-09-19T07:37:18.720 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T07:37:18.720 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.720 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.720 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.720 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.720 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.720 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.720 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.721 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.721 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.721 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.721 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.721 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.721 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.721 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.721 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.721 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.721 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.721 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.721 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.721 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.721 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.722 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.722 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.722 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.722 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.722 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.722 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.722 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.722 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.722 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.722 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.722 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.723 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.723 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.723 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.723 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.723 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.723 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.723 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.723 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.723 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.723 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.723 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.723 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.723 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.723 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.724 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.724 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.724 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.724 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.724 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.724 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.724 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.724 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.724 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.724 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.724 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.724 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.724 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.725 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.725 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.725 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.725 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.725 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.725 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.725 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.725 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.725 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.725 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.725 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.725 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.725 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.726 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.726 INFO:teuthology.orchestra.run.smithi159.stdout: "min_size": { 2024-09-19T07:37:18.726 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "min_size", 2024-09-19T07:37:18.726 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.726 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.726 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.726 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.726 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.726 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.726 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.726 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "default min_size for any created local pool", 2024-09-19T07:37:18.726 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T07:37:18.726 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.726 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.727 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.727 INFO:teuthology.orchestra.run.smithi159.stdout: "num_rep": { 2024-09-19T07:37:18.727 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "num_rep", 2024-09-19T07:37:18.727 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.727 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.727 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.727 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "3", 2024-09-19T07:37:18.727 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.727 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.728 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.728 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "default replica count for any created local pool", 2024-09-19T07:37:18.728 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.728 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.728 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.728 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.728 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num": { 2024-09-19T07:37:18.728 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "pg_num", 2024-09-19T07:37:18.728 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.728 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.728 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.728 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "128", 2024-09-19T07:37:18.728 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.728 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.729 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.729 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T07:37:18.729 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.729 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.729 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.729 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.729 INFO:teuthology.orchestra.run.smithi159.stdout: "prefix": { 2024-09-19T07:37:18.729 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "prefix", 2024-09-19T07:37:18.729 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.729 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.729 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.729 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.729 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.730 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.730 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.730 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "name prefix for any created local pool", 2024-09-19T07:37:18.730 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.730 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.730 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.730 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.730 INFO:teuthology.orchestra.run.smithi159.stdout: "subtree": { 2024-09-19T07:37:18.730 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "subtree", 2024-09-19T07:37:18.730 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.730 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.730 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.730 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "rack", 2024-09-19T07:37:18.731 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.731 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.731 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.731 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T07:37:18.731 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T07:37:18.731 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.731 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.731 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.731 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.731 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.731 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.731 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mds_autoscaler", 2024-09-19T07:37:18.731 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.731 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.732 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.732 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.732 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.732 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.732 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.732 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.732 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.732 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.732 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.732 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.732 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.732 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.733 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.733 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.733 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.733 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.733 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.733 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.733 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.733 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.733 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.733 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.734 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.734 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.734 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.734 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.734 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.734 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.734 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.734 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.734 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.734 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.734 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.734 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.734 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.734 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.735 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.735 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.735 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.735 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.735 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.735 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.735 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.735 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.735 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.735 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.735 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.735 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.735 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.735 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.736 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.736 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.736 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.736 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.736 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.736 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.736 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.736 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.736 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.736 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.736 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.736 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.736 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.736 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.737 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.737 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.737 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.737 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.737 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.737 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.737 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.737 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.737 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.737 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.737 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mirroring", 2024-09-19T07:37:18.737 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.737 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.738 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.738 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.738 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.738 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.738 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.738 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.738 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.738 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.738 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.738 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.738 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.738 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.738 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.739 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.739 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.739 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.739 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.739 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.739 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.739 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.739 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.739 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.739 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.739 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.739 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.739 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.740 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.740 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.740 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.740 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.740 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.740 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.740 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.740 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.740 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.740 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.740 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.740 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.740 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.740 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.741 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.741 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.741 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.741 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.741 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.741 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.741 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.741 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.741 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.741 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.741 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.741 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.741 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.741 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.742 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.742 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.742 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.742 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.742 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.742 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.742 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.742 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.742 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.742 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.742 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.742 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.742 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.743 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.743 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.743 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.743 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.743 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.743 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.743 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.743 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "nfs", 2024-09-19T07:37:18.743 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.743 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.743 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.743 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.743 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.743 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.744 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.744 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.744 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.744 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.744 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.744 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.744 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.744 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.744 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.744 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.744 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.744 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.744 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.745 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.745 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.745 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.745 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.745 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.745 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.745 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.745 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.745 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.745 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.745 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.745 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.745 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.746 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.746 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.746 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.746 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.746 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.746 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.746 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.746 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.746 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.746 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.746 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.746 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.746 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.746 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.747 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.747 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.747 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.747 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.747 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.747 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.747 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.747 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.747 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.747 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.747 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.747 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.747 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.748 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.748 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.748 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.748 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.748 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.748 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.748 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.748 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.748 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.748 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.748 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.748 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.748 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.749 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.749 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.749 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.749 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.749 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "orchestrator", 2024-09-19T07:37:18.749 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.749 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.749 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.749 INFO:teuthology.orchestra.run.smithi159.stdout: "fail_fs": { 2024-09-19T07:37:18.749 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "fail_fs", 2024-09-19T07:37:18.749 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.749 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.750 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.750 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.750 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.750 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.750 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.750 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T07:37:18.750 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.750 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.750 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.750 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.750 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.750 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.750 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.750 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.751 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.751 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.751 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.751 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.751 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.751 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.751 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.751 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.751 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.751 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.751 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.751 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.751 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.751 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.752 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.752 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.752 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.752 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.752 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.752 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.752 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.752 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.752 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.752 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.752 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.752 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.752 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.753 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.753 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.753 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.753 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.753 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.753 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.753 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.753 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.753 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.753 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.753 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.753 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.753 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.754 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.754 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.754 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.754 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.754 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.754 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.754 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.754 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.754 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.754 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.754 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.754 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.754 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.754 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.755 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.755 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.755 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.755 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.755 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.755 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.755 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.755 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.755 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.755 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.755 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.755 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.755 INFO:teuthology.orchestra.run.smithi159.stdout: "orchestrator": { 2024-09-19T07:37:18.755 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "orchestrator", 2024-09-19T07:37:18.756 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.756 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.756 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.756 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.756 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.756 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.756 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.756 INFO:teuthology.orchestra.run.smithi159.stdout: "cephadm", 2024-09-19T07:37:18.756 INFO:teuthology.orchestra.run.smithi159.stdout: "rook", 2024-09-19T07:37:18.756 INFO:teuthology.orchestra.run.smithi159.stdout: "test_orchestrator" 2024-09-19T07:37:18.756 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.756 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Orchestrator backend", 2024-09-19T07:37:18.756 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.757 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.757 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.757 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.757 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.757 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.757 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.757 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "osd_perf_query", 2024-09-19T07:37:18.757 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.757 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.757 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.757 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.757 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.757 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.757 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.758 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.758 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.758 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.758 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.758 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.758 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.758 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.758 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.758 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.758 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.758 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.758 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.758 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.758 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.758 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.759 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.759 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.759 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.759 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.759 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.759 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.759 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.759 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.759 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.759 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.759 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.759 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.759 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.759 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.760 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.760 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.760 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.760 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.760 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.760 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.760 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.760 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.760 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.760 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.760 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.760 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.761 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.761 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.761 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.761 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.761 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.761 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.761 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.761 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.761 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.761 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.761 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.761 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.762 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.762 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.762 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.762 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.762 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.762 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.762 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.762 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.762 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.762 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.762 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.762 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.763 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.763 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "osd_support", 2024-09-19T07:37:18.763 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.763 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.763 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.763 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.763 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.763 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.763 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.763 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.763 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.763 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.763 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.764 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.764 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.764 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.764 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.764 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.764 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.764 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.764 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.764 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.764 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.764 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.764 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.764 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.768 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.768 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.768 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.768 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.768 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.768 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.768 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.768 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.768 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.768 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.768 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.768 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "pg_autoscaler", 2024-09-19T07:37:18.768 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.769 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.769 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.769 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.769 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.769 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.769 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.769 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.769 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.769 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.769 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.769 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.769 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.769 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.769 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.769 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.770 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.770 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.770 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.770 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.770 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.770 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.770 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.770 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.770 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.770 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.770 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.770 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.770 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.770 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.771 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.771 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.771 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.771 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.771 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.771 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.771 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.771 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.771 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.771 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.771 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.771 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.771 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.772 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.772 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.772 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.772 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.772 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.772 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.772 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.772 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.772 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.772 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.772 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.772 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.772 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.772 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.773 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.773 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.773 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.773 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.773 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.773 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.773 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.773 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.773 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.773 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.773 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.773 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.773 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.774 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.774 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.774 INFO:teuthology.orchestra.run.smithi159.stdout: "sleep_interval": { 2024-09-19T07:37:18.774 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "sleep_interval", 2024-09-19T07:37:18.774 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.774 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.774 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.774 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "60", 2024-09-19T07:37:18.774 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.774 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.774 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.774 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.774 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.774 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.775 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.775 INFO:teuthology.orchestra.run.smithi159.stdout: "threshold": { 2024-09-19T07:37:18.775 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "threshold", 2024-09-19T07:37:18.775 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.775 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.775 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.775 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "3.0", 2024-09-19T07:37:18.775 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "1.0", 2024-09-19T07:37:18.775 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.775 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.775 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "scaling threshold", 2024-09-19T07:37:18.775 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T07:37:18.775 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.776 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.776 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.776 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.776 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.776 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "progress", 2024-09-19T07:37:18.776 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.776 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.776 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.776 INFO:teuthology.orchestra.run.smithi159.stdout: "allow_pg_recovery_event": { 2024-09-19T07:37:18.776 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "allow_pg_recovery_event", 2024-09-19T07:37:18.776 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.776 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.777 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.777 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.777 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.777 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.777 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.777 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T07:37:18.777 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.777 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.777 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.777 INFO:teuthology.orchestra.run.smithi159.stdout: "enabled": { 2024-09-19T07:37:18.777 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "enabled", 2024-09-19T07:37:18.777 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.777 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.778 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.778 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.778 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.778 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.778 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.778 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.778 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.778 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.778 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.778 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.778 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.778 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.778 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.779 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.779 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.779 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.779 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.779 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.779 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.779 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.779 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.779 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.779 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.779 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.779 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.779 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.780 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.780 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.780 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.780 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.780 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.780 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.780 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.780 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.780 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.780 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.780 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.780 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.781 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.781 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.781 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.781 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.781 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.781 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.781 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.781 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.781 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.781 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.781 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.781 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.782 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.782 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.782 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.782 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.782 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.782 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.782 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.782 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.782 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.782 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.782 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.782 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.783 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.783 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.783 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.783 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.783 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.783 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.783 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.783 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.783 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.783 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.783 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.783 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.783 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.784 INFO:teuthology.orchestra.run.smithi159.stdout: "max_completed_events": { 2024-09-19T07:37:18.784 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "max_completed_events", 2024-09-19T07:37:18.784 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.784 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.784 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.784 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "50", 2024-09-19T07:37:18.784 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.784 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.784 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.784 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "number of past completed events to remember", 2024-09-19T07:37:18.784 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.784 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.784 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.785 INFO:teuthology.orchestra.run.smithi159.stdout: "sleep_interval": { 2024-09-19T07:37:18.785 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "sleep_interval", 2024-09-19T07:37:18.785 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.785 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.785 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.785 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5", 2024-09-19T07:37:18.785 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.785 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.785 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.785 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how long the module is going to sleep", 2024-09-19T07:37:18.785 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.785 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.786 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.786 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.786 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.786 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.786 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "prometheus", 2024-09-19T07:37:18.786 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.786 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.786 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.786 INFO:teuthology.orchestra.run.smithi159.stdout: "cache": { 2024-09-19T07:37:18.786 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "cache", 2024-09-19T07:37:18.786 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.786 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.786 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.787 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.787 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.787 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.787 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.787 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.787 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.787 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.787 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.787 INFO:teuthology.orchestra.run.smithi159.stdout: "exclude_perf_counters": { 2024-09-19T07:37:18.787 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "exclude_perf_counters", 2024-09-19T07:37:18.787 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.787 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.788 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.788 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.788 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.788 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.788 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.788 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T07:37:18.788 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T07:37:18.788 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.788 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.788 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.788 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.788 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.789 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.789 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.789 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.789 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.789 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.789 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.789 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.789 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.789 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.789 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.789 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.789 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.789 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.789 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.790 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.790 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.790 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.790 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.790 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.790 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.790 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.790 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.790 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.790 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.790 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.790 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.790 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.791 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.791 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.791 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.791 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.791 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.791 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.791 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.791 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.791 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.791 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.791 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.791 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.791 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.791 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.792 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.792 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.792 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.792 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.792 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.792 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.792 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.792 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.792 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.792 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.792 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.792 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.793 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.793 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.793 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.793 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.793 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.793 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.793 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.793 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.793 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.793 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.793 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_stats_pools": { 2024-09-19T07:37:18.793 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rbd_stats_pools", 2024-09-19T07:37:18.793 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.793 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.794 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.794 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.794 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.794 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.794 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.794 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.794 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.794 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.794 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.794 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T07:37:18.794 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T07:37:18.794 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.795 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.795 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.795 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "300", 2024-09-19T07:37:18.795 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.795 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.795 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.795 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.795 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.795 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.795 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.795 INFO:teuthology.orchestra.run.smithi159.stdout: "scrape_interval": { 2024-09-19T07:37:18.795 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "scrape_interval", 2024-09-19T07:37:18.795 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.796 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.796 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.796 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "15.0", 2024-09-19T07:37:18.796 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.796 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.796 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.796 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.796 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.796 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.796 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.796 INFO:teuthology.orchestra.run.smithi159.stdout: "server_addr": { 2024-09-19T07:37:18.796 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_addr", 2024-09-19T07:37:18.796 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.797 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.797 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.797 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "::", 2024-09-19T07:37:18.797 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.797 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.797 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.797 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T07:37:18.797 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.797 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.797 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.797 INFO:teuthology.orchestra.run.smithi159.stdout: "server_port": { 2024-09-19T07:37:18.797 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_port", 2024-09-19T07:37:18.797 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.798 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.798 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.798 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "9283", 2024-09-19T07:37:18.798 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.798 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.798 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.798 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T07:37:18.798 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.800 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.800 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.800 INFO:teuthology.orchestra.run.smithi159.stdout: "stale_cache_strategy": { 2024-09-19T07:37:18.800 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "stale_cache_strategy", 2024-09-19T07:37:18.800 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.800 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.800 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.800 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "log", 2024-09-19T07:37:18.800 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.800 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.800 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.801 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.801 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.801 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.801 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.801 INFO:teuthology.orchestra.run.smithi159.stdout: "standby_behaviour": { 2024-09-19T07:37:18.801 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "standby_behaviour", 2024-09-19T07:37:18.801 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.801 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.801 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.801 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "default", 2024-09-19T07:37:18.801 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.801 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.801 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.802 INFO:teuthology.orchestra.run.smithi159.stdout: "default", 2024-09-19T07:37:18.802 INFO:teuthology.orchestra.run.smithi159.stdout: "error" 2024-09-19T07:37:18.802 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.802 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.802 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.802 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.802 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.802 INFO:teuthology.orchestra.run.smithi159.stdout: "standby_error_status_code": { 2024-09-19T07:37:18.802 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "standby_error_status_code", 2024-09-19T07:37:18.802 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.802 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.802 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.803 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "500", 2024-09-19T07:37:18.803 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "400", 2024-09-19T07:37:18.803 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "599", 2024-09-19T07:37:18.803 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.803 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.803 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.803 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.803 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.803 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.803 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.803 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rbd_support", 2024-09-19T07:37:18.803 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.804 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.804 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.804 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.804 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.804 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.804 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.804 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.804 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.804 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.804 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.804 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.805 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.805 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.805 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.805 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.805 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.805 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.805 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.805 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.805 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.805 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.805 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.805 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.805 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.806 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.806 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.806 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.806 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.806 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.806 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.806 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.806 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.806 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.806 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.806 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.806 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.807 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.807 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.807 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.807 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.807 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.807 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.807 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.807 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.807 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.807 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.807 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.807 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.807 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.807 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.808 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.808 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.808 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.808 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.808 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.808 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.808 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.808 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.808 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.808 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.808 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.808 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.808 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.809 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.809 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.809 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.809 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.809 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.809 INFO:teuthology.orchestra.run.smithi159.stdout: "max_concurrent_snap_create": { 2024-09-19T07:37:18.809 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "max_concurrent_snap_create", 2024-09-19T07:37:18.809 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.809 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.809 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.809 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:18.809 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.810 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.810 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.810 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.810 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.810 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.810 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.810 INFO:teuthology.orchestra.run.smithi159.stdout: "mirror_snapshot_schedule": { 2024-09-19T07:37:18.810 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T07:37:18.810 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.810 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.810 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.810 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.810 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.810 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.811 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.811 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.811 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.811 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.811 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.811 INFO:teuthology.orchestra.run.smithi159.stdout: "trash_purge_schedule": { 2024-09-19T07:37:18.811 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "trash_purge_schedule", 2024-09-19T07:37:18.811 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.811 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.811 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.811 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.812 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.812 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.812 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.812 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.812 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.812 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.812 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.812 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.812 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.812 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.812 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "restful", 2024-09-19T07:37:18.813 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.813 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.813 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.813 INFO:teuthology.orchestra.run.smithi159.stdout: "enable_auth": { 2024-09-19T07:37:18.813 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "enable_auth", 2024-09-19T07:37:18.813 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.813 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.813 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.813 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.813 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.813 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.813 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.813 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.813 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.814 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.814 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.814 INFO:teuthology.orchestra.run.smithi159.stdout: "key_file": { 2024-09-19T07:37:18.814 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "key_file", 2024-09-19T07:37:18.814 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.814 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.814 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.814 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.814 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.814 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.814 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.815 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.815 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.815 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.815 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.815 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.815 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.815 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.815 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.815 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.815 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.815 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.815 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.816 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.816 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.816 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.816 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.816 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.816 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.816 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.816 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.816 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.816 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.816 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.816 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.817 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.817 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.817 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.817 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.817 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.817 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.817 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.817 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.817 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.817 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.817 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.817 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.817 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.818 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.818 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.818 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.818 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.818 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.818 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.818 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.818 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.818 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.818 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.818 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.818 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.819 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.819 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.819 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.819 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.819 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.819 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.819 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.819 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.819 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.819 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.819 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.819 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.820 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.820 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.820 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.820 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.820 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.820 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.820 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.820 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.820 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.820 INFO:teuthology.orchestra.run.smithi159.stdout: "server_addr": { 2024-09-19T07:37:18.820 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_addr", 2024-09-19T07:37:18.820 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.820 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.821 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.821 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.821 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.821 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.821 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.821 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.821 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.821 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.821 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.821 INFO:teuthology.orchestra.run.smithi159.stdout: "server_port": { 2024-09-19T07:37:18.821 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_port", 2024-09-19T07:37:18.821 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.822 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.822 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.822 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.822 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.822 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.822 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.822 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.822 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.822 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.822 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.822 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.822 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.822 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.822 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.822 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rgw", 2024-09-19T07:37:18.823 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.823 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.823 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.823 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.823 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.823 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.823 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.823 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.823 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.823 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.823 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.823 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.823 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.823 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.824 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.824 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.824 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.824 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.824 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.824 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.824 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.824 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.824 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.824 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.824 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.824 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.825 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.825 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.825 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.825 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.825 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.825 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.825 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.825 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.825 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.825 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.825 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.825 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.825 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.825 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.826 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.826 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.826 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.826 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.826 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.826 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.826 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.826 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.826 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.826 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.826 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.826 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.826 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.827 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.827 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.827 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.827 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.827 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.827 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.827 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.827 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.827 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.827 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.827 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.827 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.827 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.827 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.828 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.828 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.828 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.828 INFO:teuthology.orchestra.run.smithi159.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T07:37:18.828 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T07:37:18.828 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.828 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.828 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.828 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5", 2024-09-19T07:37:18.828 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.828 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.828 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.828 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T07:37:18.828 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.829 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.829 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.829 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.829 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.829 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.829 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rook", 2024-09-19T07:37:18.829 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.829 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.829 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.829 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.829 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.829 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.830 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.830 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.830 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.830 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.830 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.830 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.830 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.830 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.830 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.830 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.830 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.830 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.830 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.830 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.831 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.831 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.831 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.831 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.831 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.831 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.831 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.831 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.831 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.831 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.831 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.831 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.831 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.832 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.832 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.832 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.832 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.832 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.832 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.832 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.832 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.832 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.832 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.832 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.832 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.832 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.833 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.833 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.833 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.833 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.833 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.833 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.833 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.833 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.833 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.834 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.834 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.834 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.834 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.834 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.834 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.834 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.834 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.834 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.834 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.834 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.834 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.835 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.835 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.835 INFO:teuthology.orchestra.run.smithi159.stdout: "storage_class": { 2024-09-19T07:37:18.835 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "storage_class", 2024-09-19T07:37:18.835 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.835 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.835 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.835 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "local", 2024-09-19T07:37:18.835 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.835 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.835 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.835 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T07:37:18.835 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.836 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.836 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.836 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.836 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.836 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.836 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.836 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "selftest", 2024-09-19T07:37:18.836 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.836 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.836 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.836 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.836 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.836 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.836 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.836 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.837 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.837 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.837 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.837 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.837 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.837 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.837 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.837 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.837 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.837 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.837 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.837 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.837 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.837 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.837 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.838 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.838 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.838 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.838 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.838 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.838 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.838 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.838 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.838 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.838 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.838 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.838 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.838 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.839 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.839 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.839 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.839 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.839 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.839 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.839 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.839 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.839 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.839 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.839 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.839 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.839 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.839 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.839 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.840 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.840 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.840 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.840 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.840 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.840 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.840 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.840 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.840 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.840 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.840 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.840 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.840 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.840 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.841 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.841 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.841 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.841 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.841 INFO:teuthology.orchestra.run.smithi159.stdout: "roption1": { 2024-09-19T07:37:18.841 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "roption1", 2024-09-19T07:37:18.841 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.841 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.841 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.841 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.841 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.841 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.841 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.841 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.842 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.842 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.842 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.842 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.842 INFO:teuthology.orchestra.run.smithi159.stdout: "roption2": { 2024-09-19T07:37:18.842 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "roption2", 2024-09-19T07:37:18.842 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.842 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.842 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.842 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "xyz", 2024-09-19T07:37:18.842 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.842 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.842 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.842 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.842 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.842 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.843 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.843 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.843 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption1": { 2024-09-19T07:37:18.843 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption1", 2024-09-19T07:37:18.843 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.843 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.843 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.843 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.843 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.843 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.843 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.843 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.843 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.843 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.844 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.844 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.844 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption2": { 2024-09-19T07:37:18.844 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption2", 2024-09-19T07:37:18.844 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.844 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.844 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.844 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.844 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.844 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.844 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.844 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.844 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.844 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.845 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.845 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.845 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption3": { 2024-09-19T07:37:18.845 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption3", 2024-09-19T07:37:18.845 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.845 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.845 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.845 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.845 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.845 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.845 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.845 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.846 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.846 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.846 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.846 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption4": { 2024-09-19T07:37:18.846 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption4", 2024-09-19T07:37:18.846 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.846 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.846 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.846 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.846 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.846 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.846 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.846 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.846 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.847 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.847 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.847 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption5": { 2024-09-19T07:37:18.847 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption5", 2024-09-19T07:37:18.847 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.847 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.847 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.847 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.847 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.847 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.847 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.847 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.847 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.847 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.848 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.848 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption6": { 2024-09-19T07:37:18.848 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption6", 2024-09-19T07:37:18.848 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.848 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.848 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.848 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.848 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.848 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.848 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.848 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.848 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.849 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.849 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.849 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.849 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption7": { 2024-09-19T07:37:18.849 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption7", 2024-09-19T07:37:18.849 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.849 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.849 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.849 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.849 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "1", 2024-09-19T07:37:18.849 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "42", 2024-09-19T07:37:18.849 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.849 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.849 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.850 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.850 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.850 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.850 INFO:teuthology.orchestra.run.smithi159.stdout: "testkey": { 2024-09-19T07:37:18.850 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "testkey", 2024-09-19T07:37:18.850 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.850 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.850 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.850 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.850 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.850 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.850 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.850 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.850 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.850 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.851 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.851 INFO:teuthology.orchestra.run.smithi159.stdout: "testlkey": { 2024-09-19T07:37:18.851 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "testlkey", 2024-09-19T07:37:18.851 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.851 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.851 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.851 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.851 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.851 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.851 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.851 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.851 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.851 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.851 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.852 INFO:teuthology.orchestra.run.smithi159.stdout: "testnewline": { 2024-09-19T07:37:18.852 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "testnewline", 2024-09-19T07:37:18.852 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.852 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.852 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.852 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.852 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.852 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.852 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.852 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.852 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.852 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.852 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.852 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.853 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.853 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.853 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "snap_schedule", 2024-09-19T07:37:18.853 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.853 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.853 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.853 INFO:teuthology.orchestra.run.smithi159.stdout: "allow_m_granularity": { 2024-09-19T07:37:18.853 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "allow_m_granularity", 2024-09-19T07:37:18.853 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.853 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.853 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.853 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.853 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.853 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.854 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.854 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T07:37:18.854 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.854 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.854 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.854 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.854 INFO:teuthology.orchestra.run.smithi159.stdout: "dump_on_update": { 2024-09-19T07:37:18.854 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "dump_on_update", 2024-09-19T07:37:18.854 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.854 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.854 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.854 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.854 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.854 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.855 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.855 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "dump database to debug log on update", 2024-09-19T07:37:18.855 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.855 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.855 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.855 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.855 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.855 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.855 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.855 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.855 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.855 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.855 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.855 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.856 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.856 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.856 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.856 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.856 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.856 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.856 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.856 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.856 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.856 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.856 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.856 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.857 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.857 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.857 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.857 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.857 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.857 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.857 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.857 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.857 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.857 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.858 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.858 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.858 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.858 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.858 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.858 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.858 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.858 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.858 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.858 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.858 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.858 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.859 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.859 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.859 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.859 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.859 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.859 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.859 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.860 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.860 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.860 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.860 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.860 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.860 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.860 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.860 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.861 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.861 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.861 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.861 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.861 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.861 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.861 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.861 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.861 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.862 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.862 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.862 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.862 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.862 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.862 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "stats", 2024-09-19T07:37:18.862 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.862 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.862 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.862 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.862 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.863 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.863 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.863 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.863 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.863 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.863 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.863 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.863 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.863 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.863 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.863 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.863 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.863 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.864 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.864 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.864 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.864 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.864 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.864 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.864 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.864 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.864 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.864 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.864 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.864 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.864 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.865 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.865 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.865 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.865 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.865 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.865 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.865 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.865 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.865 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.865 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.865 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.865 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.865 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.865 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.866 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.866 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.866 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.866 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.866 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.866 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.866 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.866 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.866 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.866 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.866 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.866 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.866 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.867 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.867 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.867 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.867 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.867 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.867 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.867 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.867 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.867 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.867 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.867 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.867 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.867 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.867 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.868 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.868 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.868 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.868 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.868 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "status", 2024-09-19T07:37:18.868 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.868 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.868 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.868 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.868 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.868 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.869 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.869 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.869 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.869 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.869 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.869 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.869 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.869 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.869 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.869 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.869 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.869 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.869 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.869 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.870 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.870 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.870 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.870 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.870 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.870 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.870 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.870 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.870 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.870 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.870 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.870 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.870 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.870 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.871 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.871 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.871 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.871 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.871 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.871 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.871 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.871 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.871 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.871 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.871 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.871 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.871 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.871 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.872 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.872 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.872 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.872 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.872 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.872 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.872 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.872 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.872 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.872 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.872 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.872 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.873 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.873 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.873 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.873 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.873 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.873 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.873 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.873 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.873 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.873 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.873 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.873 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.874 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.874 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.874 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.874 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.874 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "telegraf", 2024-09-19T07:37:18.874 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.874 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.874 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.874 INFO:teuthology.orchestra.run.smithi159.stdout: "address": { 2024-09-19T07:37:18.874 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "address", 2024-09-19T07:37:18.874 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.875 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.875 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.875 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T07:37:18.875 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.875 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.875 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.875 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.875 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.875 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.875 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.875 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.875 INFO:teuthology.orchestra.run.smithi159.stdout: "interval": { 2024-09-19T07:37:18.876 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "interval", 2024-09-19T07:37:18.876 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.876 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.876 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.876 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "15", 2024-09-19T07:37:18.876 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.876 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.876 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.876 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.876 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.876 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.876 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.876 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.877 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.877 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.877 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.877 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.877 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.877 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.877 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.877 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.877 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.877 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.877 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.878 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.878 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.878 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.878 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.878 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.878 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.878 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.878 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.878 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.878 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.878 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.878 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.878 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.878 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.879 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.879 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.879 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.879 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.879 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.879 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.879 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.879 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.879 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.879 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.879 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.879 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.879 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.879 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.880 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.880 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.880 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.880 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.880 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.880 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.880 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.880 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.880 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.880 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.880 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.880 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.880 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.881 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.881 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.881 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.881 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.881 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.881 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.881 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.881 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.881 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.881 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.881 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.881 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.881 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.881 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.882 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.882 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.882 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.882 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.882 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.882 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.882 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.882 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "telemetry", 2024-09-19T07:37:18.882 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.882 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.882 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.882 INFO:teuthology.orchestra.run.smithi159.stdout: "channel_basic": { 2024-09-19T07:37:18.883 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "channel_basic", 2024-09-19T07:37:18.883 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.883 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.883 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.883 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.883 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.883 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.883 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.883 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T07:37:18.883 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.883 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.883 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.883 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.883 INFO:teuthology.orchestra.run.smithi159.stdout: "channel_crash": { 2024-09-19T07:37:18.884 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "channel_crash", 2024-09-19T07:37:18.884 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.884 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.884 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.884 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.884 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.884 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.884 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.884 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T07:37:18.884 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.884 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.884 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.884 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.885 INFO:teuthology.orchestra.run.smithi159.stdout: "channel_device": { 2024-09-19T07:37:18.885 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "channel_device", 2024-09-19T07:37:18.885 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.885 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.885 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.885 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.885 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.885 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.885 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.885 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T07:37:18.885 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.885 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.885 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.885 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.886 INFO:teuthology.orchestra.run.smithi159.stdout: "channel_ident": { 2024-09-19T07:37:18.886 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "channel_ident", 2024-09-19T07:37:18.886 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.886 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.886 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.886 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.886 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.886 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.886 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.886 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T07:37:18.886 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.886 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.886 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.886 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.887 INFO:teuthology.orchestra.run.smithi159.stdout: "channel_perf": { 2024-09-19T07:37:18.887 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "channel_perf", 2024-09-19T07:37:18.887 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.887 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.887 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.887 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.887 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.887 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.887 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.887 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T07:37:18.887 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.887 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.888 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.888 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.888 INFO:teuthology.orchestra.run.smithi159.stdout: "contact": { 2024-09-19T07:37:18.888 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "contact", 2024-09-19T07:37:18.888 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.888 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.888 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.888 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.888 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.888 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.888 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.888 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.888 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.889 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.889 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.889 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.889 INFO:teuthology.orchestra.run.smithi159.stdout: "description": { 2024-09-19T07:37:18.889 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "description", 2024-09-19T07:37:18.889 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.889 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.889 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.889 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.889 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.889 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.889 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.890 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.890 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.890 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.890 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.890 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.890 INFO:teuthology.orchestra.run.smithi159.stdout: "device_url": { 2024-09-19T07:37:18.890 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "device_url", 2024-09-19T07:37:18.890 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.890 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.890 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.890 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T07:37:18.890 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.891 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.891 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.891 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.891 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.891 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.891 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.891 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.891 INFO:teuthology.orchestra.run.smithi159.stdout: "enabled": { 2024-09-19T07:37:18.891 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "enabled", 2024-09-19T07:37:18.891 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.891 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.891 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.891 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.892 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.892 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.892 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.892 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.892 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.892 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.892 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.892 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.892 INFO:teuthology.orchestra.run.smithi159.stdout: "interval": { 2024-09-19T07:37:18.892 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "interval", 2024-09-19T07:37:18.892 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.893 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.893 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.893 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "24", 2024-09-19T07:37:18.893 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "8", 2024-09-19T07:37:18.893 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.893 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.893 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.893 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.893 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.893 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.893 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.893 INFO:teuthology.orchestra.run.smithi159.stdout: "last_opt_revision": { 2024-09-19T07:37:18.893 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "last_opt_revision", 2024-09-19T07:37:18.894 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.894 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.894 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.894 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "1", 2024-09-19T07:37:18.894 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.894 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.894 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.894 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.894 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.894 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.894 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.895 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.895 INFO:teuthology.orchestra.run.smithi159.stdout: "leaderboard": { 2024-09-19T07:37:18.895 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "leaderboard", 2024-09-19T07:37:18.895 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.895 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.895 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.895 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.895 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.895 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.895 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.895 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.895 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.895 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.895 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.896 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.896 INFO:teuthology.orchestra.run.smithi159.stdout: "leaderboard_description": { 2024-09-19T07:37:18.896 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "leaderboard_description", 2024-09-19T07:37:18.896 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.896 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.896 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.896 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.896 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.896 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.896 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.896 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.896 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.897 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.897 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.897 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.897 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.897 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.897 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.897 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.897 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.897 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.897 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.897 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.897 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.897 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.898 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.898 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.898 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.898 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.898 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.898 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.898 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.898 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.898 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.898 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.898 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.898 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.898 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.899 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.899 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.899 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.899 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.899 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.899 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.899 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.899 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.899 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.899 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.899 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.899 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.899 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.900 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.900 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.900 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.900 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.900 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.900 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.900 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.900 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.900 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.900 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.900 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.900 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.900 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.900 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.901 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.901 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.901 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.901 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.901 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.901 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.901 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.901 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.901 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.901 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.901 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.901 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.901 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.902 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.902 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.902 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.902 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.902 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.902 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.902 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.902 INFO:teuthology.orchestra.run.smithi159.stdout: "organization": { 2024-09-19T07:37:18.902 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "organization", 2024-09-19T07:37:18.902 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.902 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.902 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.902 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.902 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.903 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.903 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.903 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.903 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.903 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.903 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.903 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.903 INFO:teuthology.orchestra.run.smithi159.stdout: "proxy": { 2024-09-19T07:37:18.903 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "proxy", 2024-09-19T07:37:18.903 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.903 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.903 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.903 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.904 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.904 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.904 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.904 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.904 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.904 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.904 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.904 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.904 INFO:teuthology.orchestra.run.smithi159.stdout: "url": { 2024-09-19T07:37:18.904 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "url", 2024-09-19T07:37:18.904 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.904 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.904 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.905 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T07:37:18.905 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.905 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.905 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.905 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.905 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.905 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.905 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.905 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.905 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.905 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.905 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.906 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "test_orchestrator", 2024-09-19T07:37:18.906 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.906 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.906 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.906 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.906 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.906 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.906 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.906 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.906 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.906 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.906 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.906 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.907 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.907 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.907 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.907 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.907 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.907 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.907 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.907 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.907 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.907 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.907 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.907 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.907 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.907 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.908 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.908 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.908 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.908 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.908 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.908 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.908 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.908 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.908 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.908 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.908 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.908 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.908 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.908 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.909 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.909 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.909 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.909 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.909 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.909 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.909 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.909 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.909 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.909 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.909 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.909 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.909 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.909 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.910 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.910 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.910 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.910 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.910 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.910 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.910 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.910 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.910 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.910 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.910 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.910 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.910 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.910 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.911 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.911 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.911 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.911 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.911 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.911 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.911 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.911 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.911 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "volumes", 2024-09-19T07:37:18.911 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.911 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.911 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.911 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.911 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.911 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.912 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.912 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.912 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.912 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.912 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.912 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.912 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.912 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.912 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.912 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.912 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.912 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.912 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.912 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.912 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.913 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.913 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.913 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.913 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.913 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.913 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.913 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.913 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.913 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.913 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.913 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.913 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.913 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.913 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.914 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.914 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.914 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.914 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.914 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.914 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.914 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.914 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.914 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.914 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.914 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.914 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.914 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.915 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.915 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.915 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.915 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.915 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.915 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.915 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.915 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.915 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.915 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.915 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.915 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.915 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.915 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.916 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.916 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.916 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.916 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.916 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.916 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.916 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.916 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.916 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.916 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.916 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.916 INFO:teuthology.orchestra.run.smithi159.stdout: "max_concurrent_clones": { 2024-09-19T07:37:18.916 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "max_concurrent_clones", 2024-09-19T07:37:18.917 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.917 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.917 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.917 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "4", 2024-09-19T07:37:18.917 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.917 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.917 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.917 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T07:37:18.917 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.917 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.917 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.917 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.917 INFO:teuthology.orchestra.run.smithi159.stdout: "periodic_async_work": { 2024-09-19T07:37:18.918 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "periodic_async_work", 2024-09-19T07:37:18.918 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.918 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.918 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.918 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.918 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.918 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.918 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.918 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Periodically check for async work", 2024-09-19T07:37:18.918 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.918 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.918 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.918 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.919 INFO:teuthology.orchestra.run.smithi159.stdout: "snapshot_clone_delay": { 2024-09-19T07:37:18.919 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "snapshot_clone_delay", 2024-09-19T07:37:18.919 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.919 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.919 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.919 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0", 2024-09-19T07:37:18.919 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.919 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.919 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.919 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T07:37:18.919 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.919 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.920 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.920 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.920 INFO:teuthology.orchestra.run.smithi159.stdout: "snapshot_clone_no_wait": { 2024-09-19T07:37:18.920 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T07:37:18.920 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.920 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.920 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.920 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.920 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.920 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.920 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.920 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T07:37:18.920 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.921 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.921 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.921 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.921 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.921 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.921 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.921 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "zabbix", 2024-09-19T07:37:18.921 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.921 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.921 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.921 INFO:teuthology.orchestra.run.smithi159.stdout: "discovery_interval": { 2024-09-19T07:37:18.921 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "discovery_interval", 2024-09-19T07:37:18.921 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "uint", 2024-09-19T07:37:18.921 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.922 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.922 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "100", 2024-09-19T07:37:18.922 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.922 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.922 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.922 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.922 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.922 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.922 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.922 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.922 INFO:teuthology.orchestra.run.smithi159.stdout: "identifier": { 2024-09-19T07:37:18.922 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "identifier", 2024-09-19T07:37:18.922 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.922 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.923 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.923 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.923 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.923 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.923 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.923 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.923 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.923 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.923 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.923 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.923 INFO:teuthology.orchestra.run.smithi159.stdout: "interval": { 2024-09-19T07:37:18.923 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "interval", 2024-09-19T07:37:18.923 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.923 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.923 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.924 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "60", 2024-09-19T07:37:18.924 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.924 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.924 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.924 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.924 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.924 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.924 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.924 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.924 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.924 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.924 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.924 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.925 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.925 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.925 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.925 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.925 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.925 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.925 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.925 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.925 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.925 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.925 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.925 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.925 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.926 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.926 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.926 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.926 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.926 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.926 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.926 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.926 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.926 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.926 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.926 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.926 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.926 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.926 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.927 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.927 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.927 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.927 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.927 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.927 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.927 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.927 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.927 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.927 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.927 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.927 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.927 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.927 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.928 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.928 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.928 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.928 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.928 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.928 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.928 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.928 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.928 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.928 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.928 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.928 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.928 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.929 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.929 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.929 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.929 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.929 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.929 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.929 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.929 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.929 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.929 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.929 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.929 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.929 INFO:teuthology.orchestra.run.smithi159.stdout: "zabbix_host": { 2024-09-19T07:37:18.929 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "zabbix_host", 2024-09-19T07:37:18.930 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.930 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.930 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.930 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.930 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.930 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.930 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.930 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.930 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.930 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.930 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.930 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.930 INFO:teuthology.orchestra.run.smithi159.stdout: "zabbix_port": { 2024-09-19T07:37:18.930 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "zabbix_port", 2024-09-19T07:37:18.930 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.930 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.931 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.931 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10051", 2024-09-19T07:37:18.931 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.931 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.931 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.931 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.931 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.931 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.931 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.931 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.931 INFO:teuthology.orchestra.run.smithi159.stdout: "zabbix_sender": { 2024-09-19T07:37:18.931 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "zabbix_sender", 2024-09-19T07:37:18.931 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.931 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.932 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.932 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T07:37:18.932 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.932 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.932 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.932 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.932 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.932 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.932 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.932 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.932 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.932 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.932 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-19T07:37:18.933 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.933 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.933 INFO:teuthology.orchestra.run.smithi159.stdout: "modules": [ 2024-09-19T07:37:18.933 INFO:teuthology.orchestra.run.smithi159.stdout: "cephadm", 2024-09-19T07:37:18.933 INFO:teuthology.orchestra.run.smithi159.stdout: "dashboard", 2024-09-19T07:37:18.933 INFO:teuthology.orchestra.run.smithi159.stdout: "iostat", 2024-09-19T07:37:18.933 INFO:teuthology.orchestra.run.smithi159.stdout: "nfs", 2024-09-19T07:37:18.933 INFO:teuthology.orchestra.run.smithi159.stdout: "restful" 2024-09-19T07:37:18.933 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.933 INFO:teuthology.orchestra.run.smithi159.stdout: "available_modules": [ 2024-09-19T07:37:18.934 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.934 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "alerts", 2024-09-19T07:37:18.934 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.934 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.934 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.934 INFO:teuthology.orchestra.run.smithi159.stdout: "interval": { 2024-09-19T07:37:18.934 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "interval", 2024-09-19T07:37:18.934 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.934 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.934 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.934 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "60", 2024-09-19T07:37:18.934 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.934 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.935 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.935 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T07:37:18.935 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.935 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.935 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.935 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.935 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.935 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.935 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.935 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.935 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.935 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.935 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.935 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.936 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.936 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.936 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.936 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.936 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.936 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.936 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.936 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.936 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.936 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.936 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.936 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.936 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.936 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.937 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.937 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.937 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.937 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.937 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.937 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.937 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.937 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.937 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.937 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.937 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.937 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.937 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.937 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.937 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.938 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.938 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.938 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.938 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.938 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.938 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.938 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.938 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.938 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.938 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.938 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.938 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.939 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.939 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.939 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.939 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.939 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.939 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.939 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.939 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.939 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.939 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.939 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.939 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.939 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.939 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.940 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.940 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.940 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.940 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.940 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.940 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.940 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.940 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_destination": { 2024-09-19T07:37:18.940 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_destination", 2024-09-19T07:37:18.940 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.940 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.940 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.940 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.940 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.941 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.941 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.941 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Email address to send alerts to", 2024-09-19T07:37:18.941 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.941 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.941 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.941 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.941 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_from_name": { 2024-09-19T07:37:18.941 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_from_name", 2024-09-19T07:37:18.941 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.941 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.941 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.941 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "Ceph", 2024-09-19T07:37:18.941 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.942 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.942 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.942 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Email From: name", 2024-09-19T07:37:18.942 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.942 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.942 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.942 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.942 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_host": { 2024-09-19T07:37:18.942 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_host", 2024-09-19T07:37:18.942 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.942 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.942 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.942 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.942 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.943 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.943 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.943 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "SMTP server", 2024-09-19T07:37:18.943 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.943 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.943 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.943 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.943 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_password": { 2024-09-19T07:37:18.943 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_password", 2024-09-19T07:37:18.943 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.943 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.943 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.943 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.943 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.943 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.943 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.944 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Password to authenticate with", 2024-09-19T07:37:18.944 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.944 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.944 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.944 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.944 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_port": { 2024-09-19T07:37:18.944 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_port", 2024-09-19T07:37:18.944 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.944 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.944 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.944 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "465", 2024-09-19T07:37:18.944 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.944 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.944 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.944 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "SMTP port", 2024-09-19T07:37:18.945 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.945 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.945 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.945 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.945 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_sender": { 2024-09-19T07:37:18.945 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_sender", 2024-09-19T07:37:18.945 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.945 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.945 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.945 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.945 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.945 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.945 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.945 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "SMTP envelope sender", 2024-09-19T07:37:18.945 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.946 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.946 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.946 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.946 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_ssl": { 2024-09-19T07:37:18.946 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_ssl", 2024-09-19T07:37:18.946 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.946 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.946 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.946 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.946 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.946 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.946 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.946 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T07:37:18.946 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.947 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.947 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.947 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.947 INFO:teuthology.orchestra.run.smithi159.stdout: "smtp_user": { 2024-09-19T07:37:18.947 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "smtp_user", 2024-09-19T07:37:18.947 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.947 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.947 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.947 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.947 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.947 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.947 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.947 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "User to authenticate as", 2024-09-19T07:37:18.947 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.947 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.947 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.948 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.948 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.948 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.948 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.948 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "balancer", 2024-09-19T07:37:18.948 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.948 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.948 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.948 INFO:teuthology.orchestra.run.smithi159.stdout: "active": { 2024-09-19T07:37:18.948 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "active", 2024-09-19T07:37:18.948 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.948 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.948 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.949 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.949 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.949 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.949 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.949 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T07:37:18.949 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.949 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.949 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.949 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.949 INFO:teuthology.orchestra.run.smithi159.stdout: "begin_time": { 2024-09-19T07:37:18.949 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "begin_time", 2024-09-19T07:37:18.949 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.950 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.950 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.950 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0000", 2024-09-19T07:37:18.950 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.950 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.950 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.950 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T07:37:18.950 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:37:18.950 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.950 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.950 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.950 INFO:teuthology.orchestra.run.smithi159.stdout: "begin_weekday": { 2024-09-19T07:37:18.950 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "begin_weekday", 2024-09-19T07:37:18.950 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "uint", 2024-09-19T07:37:18.951 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.951 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.951 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0", 2024-09-19T07:37:18.951 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "0", 2024-09-19T07:37:18.951 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "6", 2024-09-19T07:37:18.951 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.951 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T07:37:18.951 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:37:18.951 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.951 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.951 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.951 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_compat_max_iterations": { 2024-09-19T07:37:18.951 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "crush_compat_max_iterations", 2024-09-19T07:37:18.951 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "uint", 2024-09-19T07:37:18.952 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.952 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.952 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "25", 2024-09-19T07:37:18.952 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "1", 2024-09-19T07:37:18.952 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "250", 2024-09-19T07:37:18.952 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.952 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T07:37:18.952 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.952 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.952 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.952 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.952 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_compat_metrics": { 2024-09-19T07:37:18.952 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "crush_compat_metrics", 2024-09-19T07:37:18.953 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.953 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.953 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.953 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T07:37:18.953 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.953 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.953 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.953 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T07:37:18.953 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T07:37:18.953 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.953 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.953 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.953 INFO:teuthology.orchestra.run.smithi159.stdout: "crush_compat_step": { 2024-09-19T07:37:18.953 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "crush_compat_step", 2024-09-19T07:37:18.954 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.954 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.954 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.954 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0.5", 2024-09-19T07:37:18.954 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "0.001", 2024-09-19T07:37:18.954 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "0.999", 2024-09-19T07:37:18.954 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.954 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "aggressiveness of optimization", 2024-09-19T07:37:18.954 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T07:37:18.954 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.954 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.954 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.954 INFO:teuthology.orchestra.run.smithi159.stdout: "end_time": { 2024-09-19T07:37:18.955 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "end_time", 2024-09-19T07:37:18.955 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.955 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.955 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.955 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "2359", 2024-09-19T07:37:18.955 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.955 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.955 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.955 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T07:37:18.955 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:37:18.955 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.955 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.955 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.955 INFO:teuthology.orchestra.run.smithi159.stdout: "end_weekday": { 2024-09-19T07:37:18.955 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "end_weekday", 2024-09-19T07:37:18.956 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "uint", 2024-09-19T07:37:18.956 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.956 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.956 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0", 2024-09-19T07:37:18.956 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "0", 2024-09-19T07:37:18.956 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "6", 2024-09-19T07:37:18.956 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.956 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T07:37:18.956 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:37:18.956 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.956 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.956 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.956 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:18.956 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:18.957 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.957 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.957 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.957 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.957 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.957 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.957 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.957 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.957 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.957 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.957 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.957 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.957 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.957 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.958 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.958 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.958 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.958 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.958 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.958 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:18.958 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:18.958 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.958 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.958 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.958 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.958 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.958 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.958 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.958 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.959 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.959 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.959 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.959 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.959 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:18.959 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:18.959 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.959 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.959 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.959 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:18.959 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.959 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.959 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.959 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:18.959 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:18.960 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:18.960 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:18.960 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:18.960 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:18.960 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.960 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.960 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.960 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.960 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.960 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.960 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:18.960 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:18.960 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.960 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.961 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.961 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.961 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.961 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.961 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.961 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:18.961 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.961 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.961 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.961 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.961 INFO:teuthology.orchestra.run.smithi159.stdout: "min_score": { 2024-09-19T07:37:18.961 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "min_score", 2024-09-19T07:37:18.961 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.961 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.962 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.962 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0", 2024-09-19T07:37:18.962 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.962 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.962 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.962 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T07:37:18.962 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.962 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.962 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.962 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.962 INFO:teuthology.orchestra.run.smithi159.stdout: "mode": { 2024-09-19T07:37:18.962 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mode", 2024-09-19T07:37:18.962 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.962 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.962 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.962 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "upmap", 2024-09-19T07:37:18.963 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.963 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.963 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.963 INFO:teuthology.orchestra.run.smithi159.stdout: "crush-compat", 2024-09-19T07:37:18.963 INFO:teuthology.orchestra.run.smithi159.stdout: "none", 2024-09-19T07:37:18.963 INFO:teuthology.orchestra.run.smithi159.stdout: "read", 2024-09-19T07:37:18.963 INFO:teuthology.orchestra.run.smithi159.stdout: "upmap", 2024-09-19T07:37:18.963 INFO:teuthology.orchestra.run.smithi159.stdout: "upmap-read" 2024-09-19T07:37:18.963 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.963 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Balancer mode", 2024-09-19T07:37:18.963 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.963 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.963 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.963 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.964 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_ids": { 2024-09-19T07:37:18.964 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "pool_ids", 2024-09-19T07:37:18.964 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.964 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.964 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.964 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.964 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.964 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.964 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.964 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T07:37:18.964 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.964 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.964 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.964 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.965 INFO:teuthology.orchestra.run.smithi159.stdout: "sleep_interval": { 2024-09-19T07:37:18.965 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "sleep_interval", 2024-09-19T07:37:18.965 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.965 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.965 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.965 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "60", 2024-09-19T07:37:18.965 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.965 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.965 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.965 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T07:37:18.965 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.965 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.966 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.966 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.966 INFO:teuthology.orchestra.run.smithi159.stdout: "upmap_max_deviation": { 2024-09-19T07:37:18.966 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "upmap_max_deviation", 2024-09-19T07:37:18.966 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.966 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.966 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.966 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5", 2024-09-19T07:37:18.966 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "1", 2024-09-19T07:37:18.966 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.966 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.966 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T07:37:18.966 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T07:37:18.966 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.967 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.967 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.967 INFO:teuthology.orchestra.run.smithi159.stdout: "upmap_max_optimizations": { 2024-09-19T07:37:18.967 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "upmap_max_optimizations", 2024-09-19T07:37:18.967 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "uint", 2024-09-19T07:37:18.967 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.967 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.967 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:18.967 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.967 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.967 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.967 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T07:37:18.967 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.967 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.968 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.968 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.968 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:18.968 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.968 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:18.968 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "cephadm", 2024-09-19T07:37:18.968 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:18.968 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:18.968 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:18.968 INFO:teuthology.orchestra.run.smithi159.stdout: "agent_down_multiplier": { 2024-09-19T07:37:18.968 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "agent_down_multiplier", 2024-09-19T07:37:18.968 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.968 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.968 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.969 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "3.0", 2024-09-19T07:37:18.969 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.969 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.969 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.969 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T07:37:18.969 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.969 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.969 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.969 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.969 INFO:teuthology.orchestra.run.smithi159.stdout: "agent_refresh_rate": { 2024-09-19T07:37:18.969 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "agent_refresh_rate", 2024-09-19T07:37:18.969 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.969 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.970 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.970 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "20", 2024-09-19T07:37:18.970 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.970 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.970 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.970 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T07:37:18.970 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.970 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.970 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.970 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.970 INFO:teuthology.orchestra.run.smithi159.stdout: "agent_starting_port": { 2024-09-19T07:37:18.970 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "agent_starting_port", 2024-09-19T07:37:18.970 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.970 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.971 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.971 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "4721", 2024-09-19T07:37:18.971 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.971 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.971 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.971 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T07:37:18.971 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.971 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.971 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.971 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.971 INFO:teuthology.orchestra.run.smithi159.stdout: "allow_ptrace": { 2024-09-19T07:37:18.971 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "allow_ptrace", 2024-09-19T07:37:18.971 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.972 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.972 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.972 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.972 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.972 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.972 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.972 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T07:37:18.972 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T07:37:18.972 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.972 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.972 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.972 INFO:teuthology.orchestra.run.smithi159.stdout: "autotune_interval": { 2024-09-19T07:37:18.972 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "autotune_interval", 2024-09-19T07:37:18.972 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.973 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.973 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.973 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "600", 2024-09-19T07:37:18.973 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.973 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.973 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.973 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T07:37:18.973 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.973 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.973 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.973 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.973 INFO:teuthology.orchestra.run.smithi159.stdout: "autotune_memory_target_ratio": { 2024-09-19T07:37:18.973 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T07:37:18.973 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:18.974 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.974 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.974 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0.7", 2024-09-19T07:37:18.974 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.974 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.974 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.974 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T07:37:18.974 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.974 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.974 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.974 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.974 INFO:teuthology.orchestra.run.smithi159.stdout: "cephadm_log_destination": { 2024-09-19T07:37:18.974 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "cephadm_log_destination", 2024-09-19T07:37:18.974 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.975 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.975 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.975 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:18.975 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.975 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.975 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:18.975 INFO:teuthology.orchestra.run.smithi159.stdout: "file", 2024-09-19T07:37:18.975 INFO:teuthology.orchestra.run.smithi159.stdout: "file,syslog", 2024-09-19T07:37:18.975 INFO:teuthology.orchestra.run.smithi159.stdout: "syslog" 2024-09-19T07:37:18.975 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:18.975 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T07:37:18.975 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.976 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.976 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.976 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.976 INFO:teuthology.orchestra.run.smithi159.stdout: "cgroups_split": { 2024-09-19T07:37:18.976 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "cgroups_split", 2024-09-19T07:37:18.976 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.976 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.976 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.976 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.976 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.976 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.976 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.976 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T07:37:18.976 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.976 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.977 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.977 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.977 INFO:teuthology.orchestra.run.smithi159.stdout: "config_checks_enabled": { 2024-09-19T07:37:18.977 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "config_checks_enabled", 2024-09-19T07:37:18.977 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.977 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.977 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.977 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:18.977 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.977 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.977 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.977 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T07:37:18.977 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.977 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.978 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.978 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.978 INFO:teuthology.orchestra.run.smithi159.stdout: "config_dashboard": { 2024-09-19T07:37:18.978 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "config_dashboard", 2024-09-19T07:37:18.978 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.978 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.978 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.978 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.978 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.978 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.978 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.978 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T07:37:18.979 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.979 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.979 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.979 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.979 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_alertmanager": { 2024-09-19T07:37:18.979 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_alertmanager", 2024-09-19T07:37:18.979 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.979 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.979 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.979 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T07:37:18.979 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.979 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.979 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.979 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Prometheus container image", 2024-09-19T07:37:18.980 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.980 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.980 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.980 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.980 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_base": { 2024-09-19T07:37:18.980 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_base", 2024-09-19T07:37:18.980 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.980 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.980 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:18.980 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T07:37:18.980 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.980 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.980 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.980 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Container image name, without the tag", 2024-09-19T07:37:18.981 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.981 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.981 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.981 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.981 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_elasticsearch": { 2024-09-19T07:37:18.981 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_elasticsearch", 2024-09-19T07:37:18.981 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.981 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.981 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.981 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T07:37:18.981 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.981 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.981 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.981 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "elasticsearch container image", 2024-09-19T07:37:18.981 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.982 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.982 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.982 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.982 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_grafana": { 2024-09-19T07:37:18.982 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_grafana", 2024-09-19T07:37:18.982 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.982 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.982 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.982 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T07:37:18.982 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.982 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.982 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.982 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Prometheus container image", 2024-09-19T07:37:18.983 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.983 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.983 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.983 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.983 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_haproxy": { 2024-09-19T07:37:18.983 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_haproxy", 2024-09-19T07:37:18.983 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.983 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.983 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.983 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T07:37:18.983 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.983 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.983 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.983 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "HAproxy container image", 2024-09-19T07:37:18.983 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.984 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.984 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.984 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.984 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_jaeger_agent": { 2024-09-19T07:37:18.984 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_jaeger_agent", 2024-09-19T07:37:18.984 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.984 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.984 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.984 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T07:37:18.984 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.984 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.984 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.984 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Jaeger agent container image", 2024-09-19T07:37:18.984 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.985 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.985 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.985 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.985 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_jaeger_collector": { 2024-09-19T07:37:18.985 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_jaeger_collector", 2024-09-19T07:37:18.985 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.985 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.985 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.985 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T07:37:18.985 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.985 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.985 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.985 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Jaeger collector container image", 2024-09-19T07:37:18.985 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.986 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.986 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.986 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.986 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_jaeger_query": { 2024-09-19T07:37:18.986 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_jaeger_query", 2024-09-19T07:37:18.986 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.986 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.986 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.986 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T07:37:18.986 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.986 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.986 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.986 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Jaeger query container image", 2024-09-19T07:37:18.986 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.986 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.987 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.987 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.987 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_keepalived": { 2024-09-19T07:37:18.987 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_keepalived", 2024-09-19T07:37:18.987 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.987 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.987 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.987 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T07:37:18.987 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.987 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.987 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.987 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Keepalived container image", 2024-09-19T07:37:18.987 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.987 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.988 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.988 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.988 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_loki": { 2024-09-19T07:37:18.988 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_loki", 2024-09-19T07:37:18.988 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.988 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.988 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.988 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T07:37:18.988 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.988 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.988 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.988 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Loki container image", 2024-09-19T07:37:18.988 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.989 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.989 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.989 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.989 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_node_exporter": { 2024-09-19T07:37:18.989 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_node_exporter", 2024-09-19T07:37:18.989 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.989 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.989 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.989 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T07:37:18.989 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.989 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.989 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.989 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Prometheus container image", 2024-09-19T07:37:18.989 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.989 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.989 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.990 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.990 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_nvmeof": { 2024-09-19T07:37:18.990 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_nvmeof", 2024-09-19T07:37:18.990 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.990 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.990 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.990 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T07:37:18.990 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.990 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.990 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.990 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Nvme-of container image", 2024-09-19T07:37:18.990 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.990 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.990 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.990 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.990 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_prometheus": { 2024-09-19T07:37:18.991 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_prometheus", 2024-09-19T07:37:18.991 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.991 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.991 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.991 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T07:37:18.991 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.991 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.991 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.991 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Prometheus container image", 2024-09-19T07:37:18.991 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.991 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.991 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.991 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.991 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_promtail": { 2024-09-19T07:37:18.991 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_promtail", 2024-09-19T07:37:18.992 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.992 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.992 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.992 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T07:37:18.992 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.992 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.992 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.992 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Promtail container image", 2024-09-19T07:37:18.992 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.992 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.992 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.992 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.992 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_samba": { 2024-09-19T07:37:18.992 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_samba", 2024-09-19T07:37:18.992 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.993 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.993 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.993 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T07:37:18.993 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.993 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.993 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.993 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Samba/SMB container image", 2024-09-19T07:37:18.993 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.993 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.993 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.993 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.993 INFO:teuthology.orchestra.run.smithi159.stdout: "container_image_snmp_gateway": { 2024-09-19T07:37:18.993 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_image_snmp_gateway", 2024-09-19T07:37:18.994 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.994 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.994 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.994 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T07:37:18.994 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.994 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.994 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.994 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "SNMP Gateway container image", 2024-09-19T07:37:18.994 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.994 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.994 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.994 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.994 INFO:teuthology.orchestra.run.smithi159.stdout: "container_init": { 2024-09-19T07:37:18.995 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "container_init", 2024-09-19T07:37:18.995 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:18.995 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.995 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.995 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:18.995 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.995 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.995 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.995 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T07:37:18.995 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.995 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.995 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.995 INFO:teuthology.orchestra.run.smithi159.stdout: "daemon_cache_timeout": { 2024-09-19T07:37:18.996 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "daemon_cache_timeout", 2024-09-19T07:37:18.996 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.996 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.996 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.996 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "600", 2024-09-19T07:37:18.996 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.996 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.996 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.996 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T07:37:18.996 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.996 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.996 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.997 INFO:teuthology.orchestra.run.smithi159.stdout: "default_cephadm_command_timeout": { 2024-09-19T07:37:18.997 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T07:37:18.997 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:18.997 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.997 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.997 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "900", 2024-09-19T07:37:18.997 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.997 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.997 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.997 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T07:37:18.997 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.997 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.997 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.998 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.998 INFO:teuthology.orchestra.run.smithi159.stdout: "default_registry": { 2024-09-19T07:37:18.998 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "default_registry", 2024-09-19T07:37:18.998 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:18.998 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.998 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.998 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "docker.io", 2024-09-19T07:37:18.998 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.998 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.998 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.998 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T07:37:18.998 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.998 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.998 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.998 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:18.999 INFO:teuthology.orchestra.run.smithi159.stdout: "device_cache_timeout": { 2024-09-19T07:37:18.999 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "device_cache_timeout", 2024-09-19T07:37:18.999 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:18.999 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:18.999 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:18.999 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "1800", 2024-09-19T07:37:18.999 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:18.999 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:18.999 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:18.999 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "seconds to cache device inventory", 2024-09-19T07:37:18.999 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:18.999 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:18.999 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:18.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.000 INFO:teuthology.orchestra.run.smithi159.stdout: "device_enhanced_scan": { 2024-09-19T07:37:19.000 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "device_enhanced_scan", 2024-09-19T07:37:19.000 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.000 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.000 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.000 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.000 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.000 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.000 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.000 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T07:37:19.000 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.000 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.000 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.000 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.001 INFO:teuthology.orchestra.run.smithi159.stdout: "facts_cache_timeout": { 2024-09-19T07:37:19.001 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "facts_cache_timeout", 2024-09-19T07:37:19.001 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:19.001 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.001 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.001 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "60", 2024-09-19T07:37:19.001 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.001 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.001 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.001 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "seconds to cache host facts data", 2024-09-19T07:37:19.001 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.001 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.001 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.002 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.002 INFO:teuthology.orchestra.run.smithi159.stdout: "grafana_dashboards_path": { 2024-09-19T07:37:19.002 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "grafana_dashboards_path", 2024-09-19T07:37:19.002 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.002 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.002 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.002 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T07:37:19.002 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.002 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.002 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.002 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T07:37:19.002 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.002 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.003 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.003 INFO:teuthology.orchestra.run.smithi159.stdout: "host_check_interval": { 2024-09-19T07:37:19.003 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "host_check_interval", 2024-09-19T07:37:19.003 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:19.003 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.003 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.003 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "600", 2024-09-19T07:37:19.003 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.003 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.003 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.003 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how frequently to perform a host check", 2024-09-19T07:37:19.003 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.003 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.003 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.004 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.004 INFO:teuthology.orchestra.run.smithi159.stdout: "hw_monitoring": { 2024-09-19T07:37:19.004 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "hw_monitoring", 2024-09-19T07:37:19.004 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.004 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.004 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.004 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.004 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.004 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.004 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.004 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T07:37:19.004 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.004 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.005 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.005 INFO:teuthology.orchestra.run.smithi159.stdout: "inventory_list_all": { 2024-09-19T07:37:19.005 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "inventory_list_all", 2024-09-19T07:37:19.005 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.005 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.005 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.005 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.005 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.005 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.005 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.005 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T07:37:19.005 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.005 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.006 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.006 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.006 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.006 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.006 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.006 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.006 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.006 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.006 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.006 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.006 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.006 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.006 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.006 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.007 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.007 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.007 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.007 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.007 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.007 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.007 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.007 INFO:teuthology.orchestra.run.smithi159.stdout: "log_refresh_metadata": { 2024-09-19T07:37:19.007 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_refresh_metadata", 2024-09-19T07:37:19.007 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.007 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.007 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.007 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.007 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.008 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.008 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.008 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T07:37:19.008 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.008 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.008 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.008 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.008 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.008 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.008 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.008 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.008 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.008 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.008 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.009 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.009 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T07:37:19.009 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.009 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.009 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.009 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.009 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.009 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.009 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.009 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.009 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.009 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.009 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.010 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.010 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.010 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.010 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.010 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.010 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.010 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.010 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.010 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.010 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.010 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.010 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.010 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.010 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.011 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.011 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.011 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.011 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.011 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.011 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.011 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.011 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.011 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.011 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.011 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.011 INFO:teuthology.orchestra.run.smithi159.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T07:37:19.011 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T07:37:19.011 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.011 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.012 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.012 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.012 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.012 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.012 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.012 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T07:37:19.012 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.012 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.012 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.012 INFO:teuthology.orchestra.run.smithi159.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T07:37:19.012 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T07:37:19.012 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.012 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.012 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.013 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "*", 2024-09-19T07:37:19.013 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.013 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.013 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.013 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T07:37:19.013 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.013 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.013 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.013 INFO:teuthology.orchestra.run.smithi159.stdout: "max_count_per_host": { 2024-09-19T07:37:19.013 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "max_count_per_host", 2024-09-19T07:37:19.013 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.013 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.013 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.013 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:19.014 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.014 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.014 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.014 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "max number of daemons per service per host", 2024-09-19T07:37:19.014 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.014 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.014 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.014 INFO:teuthology.orchestra.run.smithi159.stdout: "max_osd_draining_count": { 2024-09-19T07:37:19.014 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "max_osd_draining_count", 2024-09-19T07:37:19.014 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.014 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.014 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.014 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:19.015 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.015 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.015 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.015 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T07:37:19.015 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.015 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.015 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.015 INFO:teuthology.orchestra.run.smithi159.stdout: "migration_current": { 2024-09-19T07:37:19.015 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "migration_current", 2024-09-19T07:37:19.015 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.015 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.015 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.015 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.016 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.016 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.016 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.016 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "internal - do not modify", 2024-09-19T07:37:19.016 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.016 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.016 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.016 INFO:teuthology.orchestra.run.smithi159.stdout: "mode": { 2024-09-19T07:37:19.016 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mode", 2024-09-19T07:37:19.016 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.016 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.016 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.016 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "root", 2024-09-19T07:37:19.017 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.017 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.017 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.017 INFO:teuthology.orchestra.run.smithi159.stdout: "cephadm-package", 2024-09-19T07:37:19.017 INFO:teuthology.orchestra.run.smithi159.stdout: "root" 2024-09-19T07:37:19.017 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.017 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T07:37:19.017 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.017 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.017 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.017 INFO:teuthology.orchestra.run.smithi159.stdout: "oob_default_addr": { 2024-09-19T07:37:19.017 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "oob_default_addr", 2024-09-19T07:37:19.018 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.018 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.018 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.018 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "169.254.1.1", 2024-09-19T07:37:19.018 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.018 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.018 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.018 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T07:37:19.018 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.018 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.018 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.018 INFO:teuthology.orchestra.run.smithi159.stdout: "prometheus_alerts_path": { 2024-09-19T07:37:19.018 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "prometheus_alerts_path", 2024-09-19T07:37:19.018 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.019 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.019 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.019 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T07:37:19.019 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.019 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.019 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.019 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T07:37:19.019 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.019 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.019 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.019 INFO:teuthology.orchestra.run.smithi159.stdout: "registry_insecure": { 2024-09-19T07:37:19.019 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "registry_insecure", 2024-09-19T07:37:19.020 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.020 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.020 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.020 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.020 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.020 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.020 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.020 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T07:37:19.020 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.020 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.020 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.020 INFO:teuthology.orchestra.run.smithi159.stdout: "registry_password": { 2024-09-19T07:37:19.021 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "registry_password", 2024-09-19T07:37:19.021 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.021 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.021 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.021 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.021 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.021 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.021 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.021 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T07:37:19.021 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.021 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.021 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.021 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.021 INFO:teuthology.orchestra.run.smithi159.stdout: "registry_url": { 2024-09-19T07:37:19.021 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "registry_url", 2024-09-19T07:37:19.022 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.022 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.022 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.022 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.022 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.022 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.022 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.022 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T07:37:19.022 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.022 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.022 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.022 INFO:teuthology.orchestra.run.smithi159.stdout: "registry_username": { 2024-09-19T07:37:19.022 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "registry_username", 2024-09-19T07:37:19.022 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.023 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.023 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.023 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.023 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.023 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.023 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.023 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T07:37:19.023 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.023 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.023 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.023 INFO:teuthology.orchestra.run.smithi159.stdout: "secure_monitoring_stack": { 2024-09-19T07:37:19.023 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "secure_monitoring_stack", 2024-09-19T07:37:19.023 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.023 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.024 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.024 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.024 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.024 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.024 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.024 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T07:37:19.024 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.024 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.024 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.024 INFO:teuthology.orchestra.run.smithi159.stdout: "service_discovery_port": { 2024-09-19T07:37:19.024 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "service_discovery_port", 2024-09-19T07:37:19.024 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.024 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.024 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.025 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "8765", 2024-09-19T07:37:19.025 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.025 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.025 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.025 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "cephadm service discovery port", 2024-09-19T07:37:19.025 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.025 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.025 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.025 INFO:teuthology.orchestra.run.smithi159.stdout: "ssh_config_file": { 2024-09-19T07:37:19.025 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ssh_config_file", 2024-09-19T07:37:19.025 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.025 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.026 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.026 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.026 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.026 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.026 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.026 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T07:37:19.026 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.026 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.026 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.026 INFO:teuthology.orchestra.run.smithi159.stdout: "use_agent": { 2024-09-19T07:37:19.026 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "use_agent", 2024-09-19T07:37:19.026 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.026 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.026 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.027 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.027 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.027 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.027 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.027 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T07:37:19.027 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.027 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.027 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.027 INFO:teuthology.orchestra.run.smithi159.stdout: "use_repo_digest": { 2024-09-19T07:37:19.027 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "use_repo_digest", 2024-09-19T07:37:19.027 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.027 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.027 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.028 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.028 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.028 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.028 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.028 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T07:37:19.028 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.028 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.028 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.028 INFO:teuthology.orchestra.run.smithi159.stdout: "warn_on_failed_host_check": { 2024-09-19T07:37:19.028 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "warn_on_failed_host_check", 2024-09-19T07:37:19.028 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.028 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.029 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.029 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.029 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.029 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.029 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.029 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T07:37:19.029 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.029 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.029 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.029 INFO:teuthology.orchestra.run.smithi159.stdout: "warn_on_stray_daemons": { 2024-09-19T07:37:19.029 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "warn_on_stray_daemons", 2024-09-19T07:37:19.029 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.029 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.030 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.030 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.030 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.030 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.030 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.030 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T07:37:19.030 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.030 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.030 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.030 INFO:teuthology.orchestra.run.smithi159.stdout: "warn_on_stray_hosts": { 2024-09-19T07:37:19.030 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "warn_on_stray_hosts", 2024-09-19T07:37:19.030 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.031 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.031 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.031 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.031 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.031 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.031 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.031 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T07:37:19.031 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.031 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.031 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.031 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.031 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.032 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "crash", 2024-09-19T07:37:19.032 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.032 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.032 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.032 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.032 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.032 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.032 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.032 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.032 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.032 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.032 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.032 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.032 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.033 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.033 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.033 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.033 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.033 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.033 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.033 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.033 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.033 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.033 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.033 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.033 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.033 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.034 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.034 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.034 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.034 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.034 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.034 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.034 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.034 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.034 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.034 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.035 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.035 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.035 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.035 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.035 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.035 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.035 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.035 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.035 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.035 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.035 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.035 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.035 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.036 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.036 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.036 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.036 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.036 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.036 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.036 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.036 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.036 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.036 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.036 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.036 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.036 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.037 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.037 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.037 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.037 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.037 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.037 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.037 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.037 INFO:teuthology.orchestra.run.smithi159.stdout: "retain_interval": { 2024-09-19T07:37:19.037 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "retain_interval", 2024-09-19T07:37:19.037 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:19.037 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.037 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.038 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "31536000", 2024-09-19T07:37:19.038 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.038 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.038 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.038 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T07:37:19.038 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.038 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.038 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.038 INFO:teuthology.orchestra.run.smithi159.stdout: "warn_recent_interval": { 2024-09-19T07:37:19.038 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "warn_recent_interval", 2024-09-19T07:37:19.038 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:19.038 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.038 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.039 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "1209600", 2024-09-19T07:37:19.039 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.039 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.039 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.039 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T07:37:19.039 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.039 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.039 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.039 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.039 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.039 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.039 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "dashboard", 2024-09-19T07:37:19.040 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.040 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.040 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.040 INFO:teuthology.orchestra.run.smithi159.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T07:37:19.040 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T07:37:19.040 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.040 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.040 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.040 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:19.040 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.040 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.040 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.040 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.040 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.041 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.041 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.041 INFO:teuthology.orchestra.run.smithi159.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T07:37:19.041 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T07:37:19.041 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.041 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.041 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.041 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.041 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.041 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.041 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.041 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.042 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.042 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.042 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.042 INFO:teuthology.orchestra.run.smithi159.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T07:37:19.042 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T07:37:19.042 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.042 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.042 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.042 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.042 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.042 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.042 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.042 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.043 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.043 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.043 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.043 INFO:teuthology.orchestra.run.smithi159.stdout: "AUDIT_API_ENABLED": { 2024-09-19T07:37:19.043 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T07:37:19.043 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.043 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.043 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.043 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.043 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.043 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.044 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.044 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.044 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.044 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.044 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.044 INFO:teuthology.orchestra.run.smithi159.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T07:37:19.044 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T07:37:19.044 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.044 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.044 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.044 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.044 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.044 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.045 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.045 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.045 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.045 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.045 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.045 INFO:teuthology.orchestra.run.smithi159.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T07:37:19.045 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T07:37:19.045 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.045 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.045 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.045 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.045 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.046 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.046 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.046 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.046 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.046 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.046 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.046 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T07:37:19.046 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T07:37:19.046 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.046 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.046 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.046 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.046 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.046 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.047 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.047 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.047 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.047 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.047 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.047 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.047 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T07:37:19.047 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T07:37:19.047 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.047 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.047 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.047 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.047 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.047 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.047 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.048 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.048 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.048 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.048 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.048 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T07:37:19.048 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T07:37:19.048 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.048 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.048 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.048 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.048 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.048 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.048 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.048 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.049 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.049 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.049 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.049 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.049 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T07:37:19.049 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T07:37:19.049 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.049 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.049 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.049 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.049 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.049 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.049 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.049 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.049 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.050 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.050 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.050 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.050 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T07:37:19.050 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T07:37:19.050 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.050 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.050 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.050 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.050 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.050 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.050 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.050 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.050 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.050 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.051 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.051 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.051 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T07:37:19.051 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T07:37:19.051 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.051 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.051 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.051 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.051 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.051 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.051 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.051 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.051 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.051 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.052 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.052 INFO:teuthology.orchestra.run.smithi159.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T07:37:19.052 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T07:37:19.052 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.052 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.052 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.052 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.052 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.052 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.052 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.052 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.052 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.052 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.052 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.052 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.053 INFO:teuthology.orchestra.run.smithi159.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T07:37:19.053 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T07:37:19.053 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.053 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.053 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.053 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.053 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.053 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.053 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.053 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.053 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.053 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.054 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.054 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.054 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T07:37:19.054 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T07:37:19.054 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.054 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.054 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.054 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "admin", 2024-09-19T07:37:19.054 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.054 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.054 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.054 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.054 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.054 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.055 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.055 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T07:37:19.055 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T07:37:19.055 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.055 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.055 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.055 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.055 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.055 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.055 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.055 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.055 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.055 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.056 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.056 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.056 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_API_URL": { 2024-09-19T07:37:19.056 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_API_URL", 2024-09-19T07:37:19.056 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.056 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.056 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.056 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.056 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.056 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.056 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.056 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.056 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.056 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.056 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.057 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T07:37:19.057 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T07:37:19.057 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.057 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.057 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.057 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "admin", 2024-09-19T07:37:19.057 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.057 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.057 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.057 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.057 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.057 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.057 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.058 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T07:37:19.058 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T07:37:19.058 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.058 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.058 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.058 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.058 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.058 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.058 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.058 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.058 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.058 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.058 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.058 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.059 INFO:teuthology.orchestra.run.smithi159.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T07:37:19.059 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T07:37:19.059 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.059 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.059 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.059 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.059 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.059 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.059 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.059 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.059 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.059 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.059 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.059 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.059 INFO:teuthology.orchestra.run.smithi159.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T07:37:19.060 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T07:37:19.060 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.060 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.060 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.060 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.060 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.060 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.060 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.060 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.060 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.060 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.060 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.060 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.061 INFO:teuthology.orchestra.run.smithi159.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T07:37:19.061 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T07:37:19.061 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.061 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.061 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.061 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.061 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.061 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.061 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.061 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.061 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.061 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.061 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.061 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.062 INFO:teuthology.orchestra.run.smithi159.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T07:37:19.062 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T07:37:19.062 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.062 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.062 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.062 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.062 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.062 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.062 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.062 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.062 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.062 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.062 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.062 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.062 INFO:teuthology.orchestra.run.smithi159.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T07:37:19.063 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T07:37:19.063 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.063 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.063 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.063 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.063 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.063 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.063 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.063 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.063 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.063 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.063 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.063 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.063 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T07:37:19.063 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T07:37:19.064 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.064 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.064 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.064 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.064 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.064 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.064 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.064 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.064 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.064 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.064 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.064 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.064 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T07:37:19.064 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T07:37:19.065 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.065 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.065 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.065 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.065 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.065 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.065 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.065 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.065 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.065 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.065 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.065 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T07:37:19.065 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T07:37:19.066 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.066 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.066 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.066 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.066 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.066 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.066 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.066 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.066 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.066 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.066 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.066 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T07:37:19.066 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T07:37:19.066 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.066 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.067 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.067 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.067 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.067 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.067 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.067 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.067 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.067 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.067 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.067 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T07:37:19.067 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T07:37:19.067 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.067 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.067 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.067 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.068 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.068 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.068 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.068 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.068 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.068 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.068 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.068 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.068 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T07:37:19.068 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T07:37:19.068 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.068 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.068 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.068 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.069 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.069 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.069 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.069 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.069 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.069 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.069 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.069 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T07:37:19.069 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T07:37:19.069 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.069 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.069 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.070 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.070 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.070 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.070 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.070 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.070 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.070 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.070 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.070 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.070 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T07:37:19.070 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T07:37:19.070 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.070 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.070 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.071 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.071 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.071 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.071 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.071 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.071 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.071 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.071 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.071 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T07:37:19.071 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T07:37:19.071 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.071 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.072 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.072 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T07:37:19.072 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.072 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.072 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.072 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.072 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.072 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.072 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.072 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.072 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T07:37:19.072 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T07:37:19.072 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.072 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.073 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.073 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:19.073 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.073 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.073 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.073 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.073 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.073 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.073 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.073 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.073 INFO:teuthology.orchestra.run.smithi159.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T07:37:19.073 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T07:37:19.073 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.073 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.074 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.074 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "8", 2024-09-19T07:37:19.074 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.074 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.074 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.074 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.074 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.074 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.074 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.074 INFO:teuthology.orchestra.run.smithi159.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T07:37:19.074 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T07:37:19.074 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.075 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.075 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.075 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "45", 2024-09-19T07:37:19.075 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.075 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.075 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.075 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.075 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.075 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.075 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.075 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.075 INFO:teuthology.orchestra.run.smithi159.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T07:37:19.075 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T07:37:19.075 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.076 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.076 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.076 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.076 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.076 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.076 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.076 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.076 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.076 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.076 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.077 INFO:teuthology.orchestra.run.smithi159.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T07:37:19.077 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T07:37:19.077 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.077 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.077 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.077 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "admin", 2024-09-19T07:37:19.077 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.077 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.077 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.077 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.077 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.077 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.078 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.078 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.078 INFO:teuthology.orchestra.run.smithi159.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T07:37:19.078 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T07:37:19.078 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.078 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.078 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.078 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.078 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.078 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.078 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.078 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.078 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.079 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.079 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.079 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.079 INFO:teuthology.orchestra.run.smithi159.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T07:37:19.079 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T07:37:19.079 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.079 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.079 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.079 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.079 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.079 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.079 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.079 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.079 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.079 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.080 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.080 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.080 INFO:teuthology.orchestra.run.smithi159.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T07:37:19.080 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T07:37:19.080 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.080 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.080 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.080 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0", 2024-09-19T07:37:19.080 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.080 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.080 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.081 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.081 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.081 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.081 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.081 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.081 INFO:teuthology.orchestra.run.smithi159.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T07:37:19.081 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T07:37:19.081 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.081 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.081 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.081 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:19.081 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.081 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.082 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.082 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.082 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.082 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.082 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.082 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.082 INFO:teuthology.orchestra.run.smithi159.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T07:37:19.082 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T07:37:19.082 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.082 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.082 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.082 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5", 2024-09-19T07:37:19.082 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.082 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.083 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.083 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.083 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.083 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.083 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.083 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.083 INFO:teuthology.orchestra.run.smithi159.stdout: "cross_origin_url": { 2024-09-19T07:37:19.083 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "cross_origin_url", 2024-09-19T07:37:19.083 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.083 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.083 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.083 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.083 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.083 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.083 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.084 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.084 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.084 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.084 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.084 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.084 INFO:teuthology.orchestra.run.smithi159.stdout: "crt_file": { 2024-09-19T07:37:19.084 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "crt_file", 2024-09-19T07:37:19.084 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.084 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.084 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.084 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.084 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.084 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.085 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.085 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.085 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.085 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.085 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.085 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.085 INFO:teuthology.orchestra.run.smithi159.stdout: "debug": { 2024-09-19T07:37:19.085 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "debug", 2024-09-19T07:37:19.085 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.085 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.085 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.085 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.085 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.085 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.086 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.086 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Enable/disable debug options", 2024-09-19T07:37:19.086 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.086 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.086 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.086 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.086 INFO:teuthology.orchestra.run.smithi159.stdout: "jwt_token_ttl": { 2024-09-19T07:37:19.086 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "jwt_token_ttl", 2024-09-19T07:37:19.086 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.086 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.086 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.087 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "28800", 2024-09-19T07:37:19.087 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.087 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.087 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.087 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.087 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.087 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.087 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.087 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.087 INFO:teuthology.orchestra.run.smithi159.stdout: "key_file": { 2024-09-19T07:37:19.087 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "key_file", 2024-09-19T07:37:19.087 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.087 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.088 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.088 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.088 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.088 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.088 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.088 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.088 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.088 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.088 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.088 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.088 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.088 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.089 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.089 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.089 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.089 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.089 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.089 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.089 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.089 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.089 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.089 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.089 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.089 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.089 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.090 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.090 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.090 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.090 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.090 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.090 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.090 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.090 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.090 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.090 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.090 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.090 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.090 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.091 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.091 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.091 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.091 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.091 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.091 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.091 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.091 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.091 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.091 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.091 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.091 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.091 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.092 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.092 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.092 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.092 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.092 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.092 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.092 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.092 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.092 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.092 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.092 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.092 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.092 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.093 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.093 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.093 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.093 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.093 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.093 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.093 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.093 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.093 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.093 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.093 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.093 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.093 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.094 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.094 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.094 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.094 INFO:teuthology.orchestra.run.smithi159.stdout: "motd": { 2024-09-19T07:37:19.094 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "motd", 2024-09-19T07:37:19.094 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.094 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.094 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.094 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.094 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.094 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.094 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.094 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "The message of the day", 2024-09-19T07:37:19.094 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.094 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.095 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.095 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.095 INFO:teuthology.orchestra.run.smithi159.stdout: "redirect_resolve_ip_addr": { 2024-09-19T07:37:19.095 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T07:37:19.095 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.095 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.095 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.095 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.095 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.095 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.095 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.095 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.095 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.095 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.096 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.096 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.096 INFO:teuthology.orchestra.run.smithi159.stdout: "server_addr": { 2024-09-19T07:37:19.096 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_addr", 2024-09-19T07:37:19.096 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.096 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.096 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.096 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "::", 2024-09-19T07:37:19.096 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.096 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.096 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.096 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.097 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.097 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.097 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.097 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.097 INFO:teuthology.orchestra.run.smithi159.stdout: "server_port": { 2024-09-19T07:37:19.097 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_port", 2024-09-19T07:37:19.097 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.097 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.097 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.097 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "8080", 2024-09-19T07:37:19.097 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.097 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.097 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.097 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.098 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.098 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.098 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.098 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.098 INFO:teuthology.orchestra.run.smithi159.stdout: "ssl": { 2024-09-19T07:37:19.098 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ssl", 2024-09-19T07:37:19.098 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.098 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.098 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.098 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.098 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.098 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.098 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.099 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.099 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.099 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.099 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.099 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.099 INFO:teuthology.orchestra.run.smithi159.stdout: "ssl_server_port": { 2024-09-19T07:37:19.099 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ssl_server_port", 2024-09-19T07:37:19.099 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.099 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.099 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.099 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "8443", 2024-09-19T07:37:19.099 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.099 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.099 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.100 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.100 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.100 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.100 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.100 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.100 INFO:teuthology.orchestra.run.smithi159.stdout: "standby_behaviour": { 2024-09-19T07:37:19.100 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "standby_behaviour", 2024-09-19T07:37:19.100 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.100 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.100 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.100 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "redirect", 2024-09-19T07:37:19.100 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.100 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.101 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.101 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.101 INFO:teuthology.orchestra.run.smithi159.stdout: "redirect" 2024-09-19T07:37:19.101 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.101 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.101 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.101 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.101 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.101 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.101 INFO:teuthology.orchestra.run.smithi159.stdout: "standby_error_status_code": { 2024-09-19T07:37:19.101 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "standby_error_status_code", 2024-09-19T07:37:19.101 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.101 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.101 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.102 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "500", 2024-09-19T07:37:19.102 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "400", 2024-09-19T07:37:19.102 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "599", 2024-09-19T07:37:19.102 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.102 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.102 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.102 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.102 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.102 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.102 INFO:teuthology.orchestra.run.smithi159.stdout: "url_prefix": { 2024-09-19T07:37:19.102 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "url_prefix", 2024-09-19T07:37:19.102 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.102 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.102 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.102 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.103 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.103 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.103 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.103 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.103 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.103 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.103 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.103 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.103 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.103 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.103 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.103 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "devicehealth", 2024-09-19T07:37:19.103 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.103 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.104 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.104 INFO:teuthology.orchestra.run.smithi159.stdout: "enable_monitoring": { 2024-09-19T07:37:19.104 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "enable_monitoring", 2024-09-19T07:37:19.104 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.104 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.104 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.104 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.104 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.104 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.104 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.104 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "monitor device health metrics", 2024-09-19T07:37:19.104 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.104 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.105 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.105 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.105 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.105 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.105 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.105 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.105 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.105 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.105 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.105 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.105 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.105 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.106 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.106 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.106 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.106 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.106 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.106 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.106 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.106 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.106 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.106 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.106 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.106 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.106 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.106 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.107 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.107 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.107 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.107 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.107 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.107 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.107 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.107 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.107 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.107 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.107 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.107 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.107 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.107 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.107 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.108 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.108 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.108 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.108 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.108 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.108 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.108 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.108 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.108 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.108 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.108 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.108 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.108 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.108 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.109 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.109 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.109 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.109 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.109 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.109 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.109 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.109 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.109 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.109 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.109 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.109 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.109 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.109 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.110 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.110 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.110 INFO:teuthology.orchestra.run.smithi159.stdout: "mark_out_threshold": { 2024-09-19T07:37:19.110 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mark_out_threshold", 2024-09-19T07:37:19.110 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:19.110 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.110 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.110 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "2419200", 2024-09-19T07:37:19.110 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.110 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.110 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.110 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T07:37:19.110 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.110 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.110 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.111 INFO:teuthology.orchestra.run.smithi159.stdout: "pool_name": { 2024-09-19T07:37:19.111 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "pool_name", 2024-09-19T07:37:19.111 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.111 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.111 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.111 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "device_health_metrics", 2024-09-19T07:37:19.111 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.111 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.111 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.111 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T07:37:19.111 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.111 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.111 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.111 INFO:teuthology.orchestra.run.smithi159.stdout: "retention_period": { 2024-09-19T07:37:19.112 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "retention_period", 2024-09-19T07:37:19.112 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:19.112 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.112 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.112 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "15552000", 2024-09-19T07:37:19.112 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.112 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.112 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.112 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how long to retain device health metrics", 2024-09-19T07:37:19.112 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.112 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.112 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.112 INFO:teuthology.orchestra.run.smithi159.stdout: "scrape_frequency": { 2024-09-19T07:37:19.112 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "scrape_frequency", 2024-09-19T07:37:19.113 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:19.113 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.113 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.113 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "86400", 2024-09-19T07:37:19.113 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.113 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.113 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.113 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T07:37:19.113 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.113 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.113 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.113 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.113 INFO:teuthology.orchestra.run.smithi159.stdout: "self_heal": { 2024-09-19T07:37:19.113 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "self_heal", 2024-09-19T07:37:19.114 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.114 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.114 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.114 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.114 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.114 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.114 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.114 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T07:37:19.114 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.114 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.114 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.114 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.115 INFO:teuthology.orchestra.run.smithi159.stdout: "sleep_interval": { 2024-09-19T07:37:19.115 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "sleep_interval", 2024-09-19T07:37:19.115 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:19.115 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.115 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.115 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "600", 2024-09-19T07:37:19.115 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.115 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.115 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.115 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T07:37:19.115 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.115 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.115 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.115 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.115 INFO:teuthology.orchestra.run.smithi159.stdout: "warn_threshold": { 2024-09-19T07:37:19.116 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "warn_threshold", 2024-09-19T07:37:19.116 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:19.116 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.116 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.116 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "7257600", 2024-09-19T07:37:19.116 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.116 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.116 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.116 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T07:37:19.116 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.116 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.117 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.117 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.117 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.117 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.117 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.117 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "diskprediction_local", 2024-09-19T07:37:19.117 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.117 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.117 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.117 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.117 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.117 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.117 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.117 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.118 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.118 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.118 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.118 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.118 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.118 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.118 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.118 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.118 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.118 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.118 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.118 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.118 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.118 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.118 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.119 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.119 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.119 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.119 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.119 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.119 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.119 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.119 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.119 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.119 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.119 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.119 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.119 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.119 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.119 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.120 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.120 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.120 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.120 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.120 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.120 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.120 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.120 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.120 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.120 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.120 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.121 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.121 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.121 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.121 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.121 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.121 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.121 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.121 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.121 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.121 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.121 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.121 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.121 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.121 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.121 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.122 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.122 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.122 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.122 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.122 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.122 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.122 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.122 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.122 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.122 INFO:teuthology.orchestra.run.smithi159.stdout: "predict_interval": { 2024-09-19T07:37:19.122 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "predict_interval", 2024-09-19T07:37:19.122 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.122 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.122 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.123 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "86400", 2024-09-19T07:37:19.123 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.123 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.123 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.123 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.123 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.123 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.123 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.123 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.123 INFO:teuthology.orchestra.run.smithi159.stdout: "predictor_model": { 2024-09-19T07:37:19.123 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "predictor_model", 2024-09-19T07:37:19.123 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.123 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.124 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.124 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "prophetstor", 2024-09-19T07:37:19.124 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.124 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.124 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.124 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.124 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.124 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.124 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.124 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.124 INFO:teuthology.orchestra.run.smithi159.stdout: "sleep_interval": { 2024-09-19T07:37:19.124 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "sleep_interval", 2024-09-19T07:37:19.124 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.124 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.124 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.125 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "600", 2024-09-19T07:37:19.125 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.125 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.125 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.125 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.125 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.125 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.125 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.125 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.125 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.125 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.125 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.125 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "influx", 2024-09-19T07:37:19.126 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": false, 2024-09-19T07:37:19.126 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "influxdb python module not found", 2024-09-19T07:37:19.126 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.126 INFO:teuthology.orchestra.run.smithi159.stdout: "batch_size": { 2024-09-19T07:37:19.126 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "batch_size", 2024-09-19T07:37:19.126 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.126 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.126 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.126 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5000", 2024-09-19T07:37:19.126 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.126 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.126 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.126 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T07:37:19.126 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.127 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.127 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.127 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.127 INFO:teuthology.orchestra.run.smithi159.stdout: "database": { 2024-09-19T07:37:19.127 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "database", 2024-09-19T07:37:19.127 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.127 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.127 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.127 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "ceph", 2024-09-19T07:37:19.127 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.127 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.127 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.127 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T07:37:19.127 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.128 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.128 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.128 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.128 INFO:teuthology.orchestra.run.smithi159.stdout: "hostname": { 2024-09-19T07:37:19.128 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "hostname", 2024-09-19T07:37:19.128 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.128 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.128 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.128 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.128 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.128 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.128 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.128 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "InfluxDB server hostname", 2024-09-19T07:37:19.128 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.128 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.129 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.129 INFO:teuthology.orchestra.run.smithi159.stdout: "interval": { 2024-09-19T07:37:19.129 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "interval", 2024-09-19T07:37:19.129 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:19.129 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.129 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.129 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "30", 2024-09-19T07:37:19.129 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "5", 2024-09-19T07:37:19.129 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.129 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.129 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T07:37:19.129 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.129 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.129 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.130 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.130 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.130 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.130 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.130 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.130 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.130 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.130 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.130 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.130 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.130 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.130 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.130 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.130 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.131 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.131 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.131 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.131 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.131 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.131 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.131 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.131 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.131 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.131 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.131 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.131 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.131 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.131 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.132 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.132 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.132 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.132 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.132 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.132 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.132 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.132 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.132 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.132 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.132 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.132 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.132 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.132 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.133 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.133 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.133 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.133 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.133 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.133 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.133 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.133 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.133 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.133 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.133 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.133 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.134 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.134 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.134 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.134 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.134 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.134 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.134 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.134 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.134 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.134 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.134 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.135 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.135 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.135 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.135 INFO:teuthology.orchestra.run.smithi159.stdout: "password": { 2024-09-19T07:37:19.135 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "password", 2024-09-19T07:37:19.135 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.135 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.135 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.135 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.135 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.135 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.135 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.135 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "password of InfluxDB server user", 2024-09-19T07:37:19.135 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.135 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.136 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.136 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.136 INFO:teuthology.orchestra.run.smithi159.stdout: "port": { 2024-09-19T07:37:19.136 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "port", 2024-09-19T07:37:19.136 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.136 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.136 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.136 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "8086", 2024-09-19T07:37:19.136 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.136 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.136 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.136 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "InfluxDB server port", 2024-09-19T07:37:19.136 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.136 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.136 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.137 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.137 INFO:teuthology.orchestra.run.smithi159.stdout: "ssl": { 2024-09-19T07:37:19.137 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ssl", 2024-09-19T07:37:19.137 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.137 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.137 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.137 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "false", 2024-09-19T07:37:19.137 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.137 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.137 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.137 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:37:19.137 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.137 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.137 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.137 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.138 INFO:teuthology.orchestra.run.smithi159.stdout: "threads": { 2024-09-19T07:37:19.138 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "threads", 2024-09-19T07:37:19.138 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.138 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.138 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.138 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5", 2024-09-19T07:37:19.138 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "1", 2024-09-19T07:37:19.138 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "32", 2024-09-19T07:37:19.138 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.138 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T07:37:19.138 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.138 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.138 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.138 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.139 INFO:teuthology.orchestra.run.smithi159.stdout: "username": { 2024-09-19T07:37:19.139 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "username", 2024-09-19T07:37:19.139 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.139 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.139 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.139 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.139 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.139 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.139 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.139 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "username of InfluxDB server user", 2024-09-19T07:37:19.139 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.139 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.139 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.139 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.140 INFO:teuthology.orchestra.run.smithi159.stdout: "verify_ssl": { 2024-09-19T07:37:19.140 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "verify_ssl", 2024-09-19T07:37:19.140 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.140 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.140 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.140 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "true", 2024-09-19T07:37:19.140 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.140 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.140 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.140 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:37:19.140 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.140 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.140 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.141 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.141 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.141 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.141 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.141 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "insights", 2024-09-19T07:37:19.141 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.141 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.141 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.141 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.141 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.141 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.141 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.141 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.141 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.142 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.142 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.142 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.142 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.142 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.142 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.142 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.142 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.142 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.142 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.142 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.142 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.142 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.143 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.143 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.143 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.143 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.143 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.143 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.143 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.143 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.143 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.143 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.143 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.143 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.143 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.143 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.144 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.144 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.144 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.144 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.144 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.144 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.144 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.144 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.144 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.144 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.144 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.144 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.144 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.145 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.145 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.145 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.145 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.145 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.145 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.145 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.145 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.145 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.145 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.145 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.145 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.145 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.145 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.146 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.146 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.146 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.146 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.146 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.146 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.146 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.146 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.146 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.146 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.146 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.146 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.146 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.146 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "iostat", 2024-09-19T07:37:19.147 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.147 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.147 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.147 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.147 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.147 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.147 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.147 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.147 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.147 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.147 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.147 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.147 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.147 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.147 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.148 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.148 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.148 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.148 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.148 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.148 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.148 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.148 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.148 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.148 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.148 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.148 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.148 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.148 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.149 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.149 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.149 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.149 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.149 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.149 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.149 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.149 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.149 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.149 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.149 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.149 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.149 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.150 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.150 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.150 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.150 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.150 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.150 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.150 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.150 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.150 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.150 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.150 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.150 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.150 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.150 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.151 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.151 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.151 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.151 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.151 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.151 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.151 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.151 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.151 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.151 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.151 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.151 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.151 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.151 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.152 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.152 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.152 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.152 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.152 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.152 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "k8sevents", 2024-09-19T07:37:19.152 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.152 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.152 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.152 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_event_retention_days": { 2024-09-19T07:37:19.152 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "ceph_event_retention_days", 2024-09-19T07:37:19.152 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.152 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.152 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.153 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "7", 2024-09-19T07:37:19.153 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.153 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.153 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.153 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T07:37:19.153 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.153 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.153 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.153 INFO:teuthology.orchestra.run.smithi159.stdout: "config_check_secs": { 2024-09-19T07:37:19.153 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "config_check_secs", 2024-09-19T07:37:19.153 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.153 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.154 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.154 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:19.154 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "10", 2024-09-19T07:37:19.154 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.154 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.154 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T07:37:19.154 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.154 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.154 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.154 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.154 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.154 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.154 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.155 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.155 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.155 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.155 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.155 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.155 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.155 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.155 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.155 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.155 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.155 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.155 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.155 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.155 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.156 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.156 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.156 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.156 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.156 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.156 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.156 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.156 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.156 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.156 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.156 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.156 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.156 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.157 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.157 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.157 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.157 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.157 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.157 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.157 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.157 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.157 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.157 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.157 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.157 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.157 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.157 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.158 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.158 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.158 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.158 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.158 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.158 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.158 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.158 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.158 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.158 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.158 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.158 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.158 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.158 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.158 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.159 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.159 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.159 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.159 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.159 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.159 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.159 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.159 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.159 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.159 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.159 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.159 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.159 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.159 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.160 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "localpool", 2024-09-19T07:37:19.160 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.160 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.160 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.160 INFO:teuthology.orchestra.run.smithi159.stdout: "failure_domain": { 2024-09-19T07:37:19.160 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "failure_domain", 2024-09-19T07:37:19.160 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.160 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.160 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.160 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "host", 2024-09-19T07:37:19.160 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.160 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.160 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.160 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "failure domain for any created local pool", 2024-09-19T07:37:19.160 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T07:37:19.161 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.161 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.161 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.161 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.161 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.161 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.161 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.161 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.161 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.161 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.161 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.161 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.161 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.162 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.162 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.162 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.162 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.162 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.162 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.162 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.162 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.162 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.162 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.162 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.162 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.162 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.162 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.163 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.163 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.163 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.163 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.163 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.163 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.163 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.163 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.163 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.163 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.163 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.163 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.163 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.163 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.164 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.164 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.164 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.164 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.164 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.164 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.164 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.164 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.164 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.164 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.164 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.164 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.164 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.164 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.165 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.165 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.165 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.165 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.165 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.165 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.165 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.165 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.165 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.165 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.165 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.165 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.165 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.165 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.166 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.166 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.166 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.166 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.166 INFO:teuthology.orchestra.run.smithi159.stdout: "min_size": { 2024-09-19T07:37:19.166 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "min_size", 2024-09-19T07:37:19.166 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.166 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.166 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.166 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.166 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.166 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.166 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.166 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "default min_size for any created local pool", 2024-09-19T07:37:19.166 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T07:37:19.167 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.167 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.167 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.167 INFO:teuthology.orchestra.run.smithi159.stdout: "num_rep": { 2024-09-19T07:37:19.167 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "num_rep", 2024-09-19T07:37:19.167 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.167 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.167 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.167 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "3", 2024-09-19T07:37:19.167 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.167 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.167 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.167 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "default replica count for any created local pool", 2024-09-19T07:37:19.167 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.167 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.168 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.168 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.168 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_num": { 2024-09-19T07:37:19.168 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "pg_num", 2024-09-19T07:37:19.168 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.168 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.168 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.168 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "128", 2024-09-19T07:37:19.168 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.168 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.168 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.168 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T07:37:19.168 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.168 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.168 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.169 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.169 INFO:teuthology.orchestra.run.smithi159.stdout: "prefix": { 2024-09-19T07:37:19.169 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "prefix", 2024-09-19T07:37:19.169 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.169 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.169 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.169 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.169 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.169 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.169 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.169 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "name prefix for any created local pool", 2024-09-19T07:37:19.169 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.169 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.169 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.169 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.170 INFO:teuthology.orchestra.run.smithi159.stdout: "subtree": { 2024-09-19T07:37:19.170 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "subtree", 2024-09-19T07:37:19.170 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.170 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.170 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.170 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "rack", 2024-09-19T07:37:19.170 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.170 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.170 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.170 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T07:37:19.170 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T07:37:19.170 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.170 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.170 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.170 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.171 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.171 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.171 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mds_autoscaler", 2024-09-19T07:37:19.171 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.171 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.171 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.171 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.171 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.171 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.171 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.171 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.171 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.171 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.171 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.171 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.172 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.172 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.172 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.172 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.172 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.172 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.172 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.172 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.172 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.172 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.172 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.172 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.172 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.172 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.173 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.173 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.173 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.173 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.173 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.173 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.173 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.173 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.173 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.173 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.173 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.173 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.173 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.173 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.174 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.174 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.174 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.174 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.174 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.174 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.174 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.174 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.174 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.174 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.174 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.174 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.174 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.174 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.175 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.175 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.175 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.175 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.175 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.175 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.175 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.175 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.175 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.175 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.175 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.175 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.175 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.175 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.175 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.176 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.176 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.176 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.176 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.176 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.176 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.176 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.176 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mirroring", 2024-09-19T07:37:19.176 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.176 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.176 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.176 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.176 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.176 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.177 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.177 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.177 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.177 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.177 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.177 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.177 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.177 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.177 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.177 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.177 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.177 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.177 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.178 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.178 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.178 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.178 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.178 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.178 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.178 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.178 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.178 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.178 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.178 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.178 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.178 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.178 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.178 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.179 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.179 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.179 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.179 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.179 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.179 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.179 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.179 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.179 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.179 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.179 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.179 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.179 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.179 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.179 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.180 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.180 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.180 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.180 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.180 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.180 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.180 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.180 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.180 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.180 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.180 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.180 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.180 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.180 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.181 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.181 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.181 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.181 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.181 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.181 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.181 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.181 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.181 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.181 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.181 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.181 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.181 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.181 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "nfs", 2024-09-19T07:37:19.182 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.182 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.182 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.182 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.182 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.182 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.182 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.182 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.182 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.182 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.182 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.182 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.182 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.182 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.183 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.183 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.183 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.183 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.183 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.183 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.183 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.183 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.183 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.183 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.183 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.183 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.183 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.183 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.184 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.184 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.184 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.184 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.184 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.184 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.184 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.184 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.184 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.184 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.184 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.184 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.184 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.184 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.185 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.185 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.185 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.185 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.185 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.185 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.185 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.185 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.185 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.185 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.185 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.185 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.185 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.185 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.185 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.186 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.186 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.186 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.186 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.186 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.186 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.186 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.186 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.186 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.186 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.186 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.186 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.186 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.186 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.186 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.187 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.187 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.187 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.187 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.187 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "orchestrator", 2024-09-19T07:37:19.187 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.187 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.187 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.187 INFO:teuthology.orchestra.run.smithi159.stdout: "fail_fs": { 2024-09-19T07:37:19.187 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "fail_fs", 2024-09-19T07:37:19.187 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.187 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.187 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.187 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.188 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.188 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.188 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.188 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T07:37:19.188 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.188 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.188 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.188 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.188 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.188 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.188 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.188 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.188 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.188 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.189 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.189 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.189 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.189 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.189 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.189 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.189 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.189 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.189 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.189 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.189 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.189 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.189 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.189 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.190 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.190 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.190 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.190 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.190 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.190 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.190 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.190 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.190 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.190 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.190 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.190 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.190 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.191 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.191 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.191 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.191 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.191 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.191 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.191 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.191 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.191 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.191 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.191 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.191 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.191 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.191 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.192 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.192 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.192 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.192 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.192 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.192 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.192 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.192 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.192 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.192 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.192 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.192 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.192 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.192 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.193 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.193 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.193 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.193 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.193 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.193 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.193 INFO:teuthology.orchestra.run.smithi159.stdout: "orchestrator": { 2024-09-19T07:37:19.193 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "orchestrator", 2024-09-19T07:37:19.193 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.193 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.193 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.193 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.193 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.194 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.194 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.194 INFO:teuthology.orchestra.run.smithi159.stdout: "cephadm", 2024-09-19T07:37:19.194 INFO:teuthology.orchestra.run.smithi159.stdout: "rook", 2024-09-19T07:37:19.194 INFO:teuthology.orchestra.run.smithi159.stdout: "test_orchestrator" 2024-09-19T07:37:19.194 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.194 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Orchestrator backend", 2024-09-19T07:37:19.194 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.194 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.194 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.194 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.194 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.194 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.195 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "osd_perf_query", 2024-09-19T07:37:19.195 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.195 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.195 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.195 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.195 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.195 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.195 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.195 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.195 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.195 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.195 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.195 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.196 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.196 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.196 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.196 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.196 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.196 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.196 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.196 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.196 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.196 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.196 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.197 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.197 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.197 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.197 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.197 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.197 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.197 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.197 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.197 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.197 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.197 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.197 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.197 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.197 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.198 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.198 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.198 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.198 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.198 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.198 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.198 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.198 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.198 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.198 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.198 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.198 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.198 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.198 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.198 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.199 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.199 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.199 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.199 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.199 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.199 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.199 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.199 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.199 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.199 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.199 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.199 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.199 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.200 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.200 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.200 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.200 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.200 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.200 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.200 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.200 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.200 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.200 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.200 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "osd_support", 2024-09-19T07:37:19.200 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.200 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.200 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.201 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.201 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.201 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.201 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.201 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.201 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.201 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.201 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.201 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.201 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.201 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.201 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.201 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.201 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.202 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.202 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.202 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.202 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.202 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.202 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.202 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.202 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.202 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.202 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.202 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.202 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.202 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.202 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.202 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.202 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.203 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.203 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.203 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.203 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.203 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.203 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.203 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.203 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.203 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.203 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.203 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.203 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.203 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.203 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.203 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.203 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.204 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.204 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.204 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.204 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.204 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.204 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.204 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.204 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.204 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.204 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.204 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.204 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.204 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.205 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.205 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.205 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.205 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.205 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.205 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.205 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.205 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.205 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.205 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.205 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.205 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.205 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.205 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.206 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "pg_autoscaler", 2024-09-19T07:37:19.206 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.206 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.206 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.206 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.206 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.206 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.206 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.206 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.206 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.206 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.206 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.206 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.206 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.207 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.207 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.207 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.207 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.207 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.207 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.207 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.207 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.207 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.207 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.207 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.207 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.207 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.207 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.208 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.208 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.208 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.208 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.208 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.208 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.208 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.208 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.208 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.208 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.208 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.208 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.209 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.209 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.209 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.209 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.209 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.209 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.209 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.209 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.209 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.209 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.209 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.209 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.209 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.210 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.210 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.210 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.210 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.212 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.212 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.212 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.212 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.212 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.212 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.213 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.213 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.213 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.213 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.213 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.213 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.213 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.213 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.213 INFO:teuthology.orchestra.run.smithi159.stdout: "sleep_interval": { 2024-09-19T07:37:19.213 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "sleep_interval", 2024-09-19T07:37:19.213 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:19.213 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.214 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.214 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "60", 2024-09-19T07:37:19.214 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.214 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.214 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.214 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.214 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.214 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.214 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.214 INFO:teuthology.orchestra.run.smithi159.stdout: "threshold": { 2024-09-19T07:37:19.214 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "threshold", 2024-09-19T07:37:19.214 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:19.214 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.215 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.215 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "3.0", 2024-09-19T07:37:19.215 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "1.0", 2024-09-19T07:37:19.215 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.215 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.215 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "scaling threshold", 2024-09-19T07:37:19.215 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T07:37:19.215 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.215 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.215 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.215 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.215 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.216 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "progress", 2024-09-19T07:37:19.216 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.216 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.216 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.216 INFO:teuthology.orchestra.run.smithi159.stdout: "allow_pg_recovery_event": { 2024-09-19T07:37:19.216 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "allow_pg_recovery_event", 2024-09-19T07:37:19.216 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.216 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.216 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.216 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.216 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.216 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.216 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.217 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T07:37:19.217 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.217 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.217 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.217 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.217 INFO:teuthology.orchestra.run.smithi159.stdout: "enabled": { 2024-09-19T07:37:19.217 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "enabled", 2024-09-19T07:37:19.217 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.217 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.217 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.217 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.217 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.217 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.217 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.218 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.218 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.218 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.218 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.218 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.218 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.218 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.218 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.218 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.218 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.218 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.218 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.218 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.218 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.219 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.219 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.219 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.219 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.219 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.219 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.219 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.219 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.219 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.219 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.219 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.219 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.219 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.219 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.220 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.220 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.220 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.220 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.220 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.220 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.220 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.220 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.220 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.220 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.220 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.220 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.220 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.221 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.221 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.221 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.221 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.221 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.221 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.221 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.221 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.221 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.221 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.221 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.221 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.221 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.221 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.222 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.222 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.222 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.222 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.222 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.222 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.222 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.222 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.222 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.222 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.222 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.222 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.222 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.222 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.223 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.223 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.223 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.223 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.223 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.223 INFO:teuthology.orchestra.run.smithi159.stdout: "max_completed_events": { 2024-09-19T07:37:19.223 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "max_completed_events", 2024-09-19T07:37:19.223 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.223 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.223 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.223 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "50", 2024-09-19T07:37:19.223 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.223 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.224 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.224 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "number of past completed events to remember", 2024-09-19T07:37:19.224 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.224 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.224 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.224 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.224 INFO:teuthology.orchestra.run.smithi159.stdout: "sleep_interval": { 2024-09-19T07:37:19.224 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "sleep_interval", 2024-09-19T07:37:19.224 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:19.224 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.224 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.224 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5", 2024-09-19T07:37:19.224 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.225 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.225 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.225 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "how long the module is going to sleep", 2024-09-19T07:37:19.225 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.225 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.225 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.225 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.225 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.225 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.225 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.225 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "prometheus", 2024-09-19T07:37:19.225 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.225 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.226 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.226 INFO:teuthology.orchestra.run.smithi159.stdout: "cache": { 2024-09-19T07:37:19.226 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "cache", 2024-09-19T07:37:19.226 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.226 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.226 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.226 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.226 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.226 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.226 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.226 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.226 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.226 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.226 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.227 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.227 INFO:teuthology.orchestra.run.smithi159.stdout: "exclude_perf_counters": { 2024-09-19T07:37:19.227 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "exclude_perf_counters", 2024-09-19T07:37:19.227 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.227 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.227 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.227 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.227 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.227 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.227 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.227 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T07:37:19.227 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T07:37:19.227 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.228 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.228 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.228 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.228 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.228 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.228 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.228 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.228 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.228 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.228 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.228 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.228 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.228 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.229 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.229 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.229 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.229 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.229 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.229 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.229 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.229 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.229 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.229 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.229 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.229 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.229 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.230 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.230 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.230 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.230 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.230 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.230 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.230 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.230 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.230 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.230 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.230 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.230 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.230 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.230 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.231 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.231 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.231 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.231 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.231 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.231 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.231 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.231 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.231 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.231 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.231 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.231 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.231 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.232 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.232 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.232 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.232 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.232 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.232 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.232 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.232 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.232 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.232 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.232 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.232 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.232 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.232 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.233 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.233 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.233 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.233 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.233 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.233 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_stats_pools": { 2024-09-19T07:37:19.233 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rbd_stats_pools", 2024-09-19T07:37:19.233 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.233 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.233 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.233 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.233 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.233 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.234 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.234 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.234 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.234 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.234 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.234 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.234 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T07:37:19.234 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T07:37:19.234 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.234 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.235 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.235 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "300", 2024-09-19T07:37:19.235 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.235 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.235 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.235 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.235 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.235 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.235 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.235 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.235 INFO:teuthology.orchestra.run.smithi159.stdout: "scrape_interval": { 2024-09-19T07:37:19.235 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "scrape_interval", 2024-09-19T07:37:19.235 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:19.235 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.236 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.236 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "15.0", 2024-09-19T07:37:19.236 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.236 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.236 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.236 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.236 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.236 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.236 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.236 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.236 INFO:teuthology.orchestra.run.smithi159.stdout: "server_addr": { 2024-09-19T07:37:19.236 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_addr", 2024-09-19T07:37:19.236 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.236 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.237 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.237 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "::", 2024-09-19T07:37:19.237 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.237 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.237 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.237 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T07:37:19.237 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.237 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.237 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.237 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.237 INFO:teuthology.orchestra.run.smithi159.stdout: "server_port": { 2024-09-19T07:37:19.237 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_port", 2024-09-19T07:37:19.237 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.237 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.238 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.238 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "9283", 2024-09-19T07:37:19.238 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.238 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.238 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.238 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T07:37:19.238 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.238 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.238 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.238 INFO:teuthology.orchestra.run.smithi159.stdout: "stale_cache_strategy": { 2024-09-19T07:37:19.238 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "stale_cache_strategy", 2024-09-19T07:37:19.238 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.238 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.238 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.239 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "log", 2024-09-19T07:37:19.239 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.239 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.239 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.239 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.239 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.239 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.239 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.239 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.239 INFO:teuthology.orchestra.run.smithi159.stdout: "standby_behaviour": { 2024-09-19T07:37:19.239 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "standby_behaviour", 2024-09-19T07:37:19.239 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.239 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.239 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.240 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "default", 2024-09-19T07:37:19.240 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.240 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.240 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.240 INFO:teuthology.orchestra.run.smithi159.stdout: "default", 2024-09-19T07:37:19.240 INFO:teuthology.orchestra.run.smithi159.stdout: "error" 2024-09-19T07:37:19.240 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.240 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.240 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.240 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.240 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.240 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.241 INFO:teuthology.orchestra.run.smithi159.stdout: "standby_error_status_code": { 2024-09-19T07:37:19.241 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "standby_error_status_code", 2024-09-19T07:37:19.241 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.241 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.241 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.241 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "500", 2024-09-19T07:37:19.241 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "400", 2024-09-19T07:37:19.241 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "599", 2024-09-19T07:37:19.241 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.241 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.241 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.241 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.241 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.241 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.242 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.242 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.242 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.242 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rbd_support", 2024-09-19T07:37:19.242 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.242 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.242 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.242 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.242 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.242 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.242 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.242 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.242 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.242 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.243 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.243 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.243 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.243 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.243 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.243 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.243 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.243 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.243 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.243 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.243 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.243 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.243 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.243 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.244 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.244 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.244 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.244 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.244 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.244 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.244 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.244 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.244 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.244 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.244 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.244 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.244 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.245 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.245 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.245 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.245 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.245 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.245 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.245 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.245 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.245 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.245 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.245 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.245 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.245 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.245 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.246 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.246 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.246 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.246 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.246 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.246 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.246 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.246 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.246 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.246 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.246 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.246 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.246 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.247 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.247 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.247 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.247 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.247 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.247 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.247 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.247 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.247 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.247 INFO:teuthology.orchestra.run.smithi159.stdout: "max_concurrent_snap_create": { 2024-09-19T07:37:19.247 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "max_concurrent_snap_create", 2024-09-19T07:37:19.247 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.247 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.247 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.248 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10", 2024-09-19T07:37:19.248 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.248 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.248 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.248 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.248 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.248 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.248 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.248 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.248 INFO:teuthology.orchestra.run.smithi159.stdout: "mirror_snapshot_schedule": { 2024-09-19T07:37:19.248 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T07:37:19.248 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.248 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.248 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.249 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.249 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.249 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.249 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.249 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.249 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.249 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.249 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.249 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.249 INFO:teuthology.orchestra.run.smithi159.stdout: "trash_purge_schedule": { 2024-09-19T07:37:19.249 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "trash_purge_schedule", 2024-09-19T07:37:19.249 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.249 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.249 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.250 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.250 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.250 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.250 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.250 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.250 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.250 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.250 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.250 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.250 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.250 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.250 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.250 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "restful", 2024-09-19T07:37:19.250 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.251 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.251 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.251 INFO:teuthology.orchestra.run.smithi159.stdout: "enable_auth": { 2024-09-19T07:37:19.251 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "enable_auth", 2024-09-19T07:37:19.251 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.251 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.251 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.251 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.251 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.251 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.251 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.251 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.251 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.251 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.251 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.252 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.252 INFO:teuthology.orchestra.run.smithi159.stdout: "key_file": { 2024-09-19T07:37:19.252 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "key_file", 2024-09-19T07:37:19.252 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.252 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.252 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.252 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.252 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.252 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.252 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.252 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.252 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.252 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.252 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.253 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.253 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.253 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.253 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.253 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.253 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.253 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.253 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.253 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.253 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.253 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.253 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.253 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.253 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.254 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.254 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.254 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.254 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.254 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.254 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.254 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.254 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.254 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.254 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.254 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.254 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.254 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.254 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.255 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.255 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.255 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.255 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.255 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.255 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.255 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.255 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.255 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.255 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.255 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.255 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.255 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.255 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.255 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.256 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.256 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.256 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.256 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.256 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.256 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.256 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.256 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.256 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.256 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.256 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.256 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.256 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.256 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.257 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.257 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.257 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.257 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.257 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.257 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.257 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.257 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.257 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.257 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.257 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.257 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.257 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.258 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.258 INFO:teuthology.orchestra.run.smithi159.stdout: "server_addr": { 2024-09-19T07:37:19.258 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_addr", 2024-09-19T07:37:19.258 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.258 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.258 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.258 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.258 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.258 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.258 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.258 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.258 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.258 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.258 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.259 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.259 INFO:teuthology.orchestra.run.smithi159.stdout: "server_port": { 2024-09-19T07:37:19.259 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "server_port", 2024-09-19T07:37:19.259 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.259 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.259 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.259 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.259 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.259 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.259 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.259 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.259 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.259 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.259 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.260 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.260 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.260 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.260 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.260 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rgw", 2024-09-19T07:37:19.260 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.260 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.260 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.260 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.260 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.260 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.261 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.261 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.261 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.261 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.261 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.261 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.261 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.261 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.261 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.261 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.261 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.261 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.261 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.261 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.262 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.262 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.262 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.262 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.262 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.262 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.262 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.262 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.262 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.262 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.262 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.262 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.262 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.263 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.263 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.263 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.263 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.263 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.263 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.263 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.263 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.263 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.263 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.263 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.263 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.263 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.264 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.264 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.264 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.264 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.264 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.264 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.264 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.264 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.264 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.264 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.264 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.264 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.264 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.265 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.265 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.265 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.265 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.265 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.265 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.265 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.265 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.265 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.265 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.265 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.265 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.265 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.265 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.266 INFO:teuthology.orchestra.run.smithi159.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T07:37:19.266 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T07:37:19.266 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.266 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.266 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.266 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "5", 2024-09-19T07:37:19.266 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.266 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.266 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.266 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T07:37:19.266 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.266 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.266 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.267 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.267 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.267 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.267 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.267 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rook", 2024-09-19T07:37:19.267 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.267 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.267 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.267 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.267 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.267 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.267 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.267 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.267 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.268 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.268 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.268 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.268 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.268 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.268 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.268 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.268 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.268 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.268 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.268 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.268 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.268 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.269 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.269 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.269 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.269 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.269 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.269 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.269 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.269 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.269 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.269 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.269 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.269 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.269 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.269 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.270 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.270 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.270 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.270 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.270 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.270 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.270 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.270 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.270 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.270 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.270 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.270 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.270 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.270 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.271 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.271 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.271 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.271 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.271 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.271 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.271 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.271 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.271 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.271 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.271 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.271 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.271 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.271 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.272 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.272 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.272 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.272 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.272 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.272 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.272 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.272 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.272 INFO:teuthology.orchestra.run.smithi159.stdout: "storage_class": { 2024-09-19T07:37:19.272 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "storage_class", 2024-09-19T07:37:19.272 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.272 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.272 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.273 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "local", 2024-09-19T07:37:19.273 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.273 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.273 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.273 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T07:37:19.273 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.273 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.273 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.273 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.273 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.273 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.273 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.273 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "selftest", 2024-09-19T07:37:19.273 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.274 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.274 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.274 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.274 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.274 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.274 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.274 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.274 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.274 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.274 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.274 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.274 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.274 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.274 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.275 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.275 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.275 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.275 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.275 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.275 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.275 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.275 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.275 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.275 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.275 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.275 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.275 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.275 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.276 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.276 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.276 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.276 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.276 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.276 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.276 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.276 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.276 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.276 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.276 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.276 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.276 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.277 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.277 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.277 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.277 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.277 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.277 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.277 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.277 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.277 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.277 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.277 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.277 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.277 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.278 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.278 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.278 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.278 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.278 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.278 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.278 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.278 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.278 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.278 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.278 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.278 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.278 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.278 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.279 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.279 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.279 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.279 INFO:teuthology.orchestra.run.smithi159.stdout: "roption1": { 2024-09-19T07:37:19.279 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "roption1", 2024-09-19T07:37:19.279 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.279 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.279 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.279 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.279 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.279 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.279 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.280 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.280 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.280 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.280 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.280 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.280 INFO:teuthology.orchestra.run.smithi159.stdout: "roption2": { 2024-09-19T07:37:19.280 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "roption2", 2024-09-19T07:37:19.280 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.280 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.280 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.280 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "xyz", 2024-09-19T07:37:19.280 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.280 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.280 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.281 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.281 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.281 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.281 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.281 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.281 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption1": { 2024-09-19T07:37:19.281 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption1", 2024-09-19T07:37:19.281 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.281 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.281 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.281 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.281 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.281 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.281 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.282 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.282 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.282 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.282 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.282 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption2": { 2024-09-19T07:37:19.282 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption2", 2024-09-19T07:37:19.282 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.282 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.282 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.282 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.282 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.282 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.282 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.283 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.283 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.283 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.283 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.283 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.283 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption3": { 2024-09-19T07:37:19.283 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption3", 2024-09-19T07:37:19.283 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "float", 2024-09-19T07:37:19.283 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.283 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.283 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.283 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.283 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.283 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.284 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.284 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.284 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.284 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.284 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption4": { 2024-09-19T07:37:19.284 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption4", 2024-09-19T07:37:19.284 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.284 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.284 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.284 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.284 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.284 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.285 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.285 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.285 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.285 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.285 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.285 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption5": { 2024-09-19T07:37:19.285 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption5", 2024-09-19T07:37:19.285 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.285 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.285 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.285 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.285 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.285 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.286 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.286 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.286 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.286 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.286 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.286 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.286 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption6": { 2024-09-19T07:37:19.286 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption6", 2024-09-19T07:37:19.286 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.286 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.286 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.286 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.286 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.286 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.287 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.287 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.287 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.287 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.287 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.287 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.287 INFO:teuthology.orchestra.run.smithi159.stdout: "rwoption7": { 2024-09-19T07:37:19.287 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rwoption7", 2024-09-19T07:37:19.287 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.287 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.287 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.287 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.287 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "1", 2024-09-19T07:37:19.287 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "42", 2024-09-19T07:37:19.288 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.288 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.288 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.288 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.288 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.288 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.288 INFO:teuthology.orchestra.run.smithi159.stdout: "testkey": { 2024-09-19T07:37:19.288 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "testkey", 2024-09-19T07:37:19.288 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.288 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.288 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.288 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.288 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.288 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.289 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.289 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.289 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.289 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.289 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.289 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.289 INFO:teuthology.orchestra.run.smithi159.stdout: "testlkey": { 2024-09-19T07:37:19.289 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "testlkey", 2024-09-19T07:37:19.289 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.289 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.289 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.289 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.289 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.289 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.290 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.290 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.290 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.290 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.290 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.290 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.290 INFO:teuthology.orchestra.run.smithi159.stdout: "testnewline": { 2024-09-19T07:37:19.290 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "testnewline", 2024-09-19T07:37:19.290 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.290 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.290 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.290 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.290 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.291 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.291 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.291 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.291 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.291 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.291 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.291 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.291 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.291 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.291 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.291 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "snap_schedule", 2024-09-19T07:37:19.291 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.291 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.291 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.291 INFO:teuthology.orchestra.run.smithi159.stdout: "allow_m_granularity": { 2024-09-19T07:37:19.292 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "allow_m_granularity", 2024-09-19T07:37:19.292 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.292 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.292 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.292 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.292 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.292 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.292 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.292 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T07:37:19.292 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.292 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.292 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.292 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.293 INFO:teuthology.orchestra.run.smithi159.stdout: "dump_on_update": { 2024-09-19T07:37:19.293 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "dump_on_update", 2024-09-19T07:37:19.293 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.293 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.293 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.293 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.293 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.293 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.293 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.293 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "dump database to debug log on update", 2024-09-19T07:37:19.293 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.293 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.293 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.293 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.294 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.294 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.294 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.294 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.294 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.294 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.294 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.294 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.294 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.294 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.294 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.294 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.294 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.294 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.295 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.295 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.295 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.295 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.295 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.295 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.295 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.295 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.295 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.295 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.295 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.295 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.295 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.296 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.296 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.296 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.296 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.296 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.296 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.296 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.296 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.296 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.296 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.296 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.296 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.296 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.297 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.297 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.297 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.297 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.297 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.297 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.297 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.297 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.297 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.297 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.297 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.297 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.297 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.297 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.298 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.298 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.298 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.298 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.298 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.298 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.298 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.298 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.298 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.298 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.298 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.298 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.298 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.298 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.299 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.299 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.299 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.299 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.299 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.299 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "stats", 2024-09-19T07:37:19.299 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.299 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.299 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.299 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.299 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.299 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.299 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.299 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.300 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.300 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.300 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.300 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.300 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.300 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.300 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.300 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.300 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.300 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.300 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.300 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.300 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.301 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.301 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.301 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.301 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.301 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.301 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.301 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.301 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.301 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.301 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.301 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.301 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.302 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.302 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.302 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.302 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.302 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.302 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.302 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.302 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.302 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.302 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.302 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.302 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.302 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.302 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.303 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.303 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.303 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.303 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.303 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.303 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.303 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.303 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.303 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.303 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.303 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.303 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.303 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.304 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.304 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.304 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.304 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.304 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.304 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.304 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.304 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.304 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.304 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.304 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.304 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.304 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.305 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.305 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.305 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.305 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "status", 2024-09-19T07:37:19.305 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.305 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.305 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.305 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.305 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.305 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.305 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.305 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.305 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.305 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.306 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.306 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.306 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.306 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.306 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.306 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.306 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.306 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.306 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.306 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.306 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.306 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.306 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.307 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.307 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.307 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.307 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.307 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.307 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.307 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.307 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.307 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.307 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.307 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.307 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.307 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.308 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.308 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.308 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.308 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.308 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.308 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.308 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.308 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.308 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.308 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.308 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.308 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.308 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.308 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.308 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.309 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.309 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.309 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.309 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.309 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.309 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.309 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.309 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.309 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.309 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.309 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.309 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.309 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.309 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.310 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.310 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.310 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.310 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.310 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.310 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.310 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.310 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.310 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.310 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.310 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "telegraf", 2024-09-19T07:37:19.310 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.310 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.310 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.310 INFO:teuthology.orchestra.run.smithi159.stdout: "address": { 2024-09-19T07:37:19.311 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "address", 2024-09-19T07:37:19.311 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.311 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.311 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.311 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T07:37:19.311 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.311 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.311 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.311 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.311 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.311 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.311 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.311 INFO:teuthology.orchestra.run.smithi159.stdout: "interval": { 2024-09-19T07:37:19.312 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "interval", 2024-09-19T07:37:19.312 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:19.312 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.312 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.312 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "15", 2024-09-19T07:37:19.312 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.312 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.312 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.312 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.312 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.312 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.312 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.313 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.313 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.313 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.313 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.313 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.313 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.313 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.313 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.313 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.313 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.313 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.313 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.313 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.314 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.314 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.314 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.314 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.314 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.314 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.314 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.314 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.314 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.314 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.314 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.314 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.314 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.315 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.315 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.315 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.315 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.315 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.315 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.315 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.315 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.315 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.315 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.315 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.315 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.315 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.315 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.315 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.316 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.316 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.316 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.316 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.316 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.316 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.316 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.316 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.316 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.316 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.316 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.316 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.317 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.317 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.317 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.317 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.317 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.317 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.317 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.317 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.317 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.317 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.317 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.317 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.317 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.317 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.318 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.318 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.318 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.318 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.318 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.318 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.318 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "telemetry", 2024-09-19T07:37:19.318 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.318 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.318 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.318 INFO:teuthology.orchestra.run.smithi159.stdout: "channel_basic": { 2024-09-19T07:37:19.318 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "channel_basic", 2024-09-19T07:37:19.318 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.318 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.319 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.319 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.319 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.319 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.319 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.319 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T07:37:19.319 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.319 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.319 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.319 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.319 INFO:teuthology.orchestra.run.smithi159.stdout: "channel_crash": { 2024-09-19T07:37:19.319 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "channel_crash", 2024-09-19T07:37:19.319 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.319 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.320 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.320 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.320 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.320 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.320 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.320 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T07:37:19.320 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.320 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.320 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.320 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.320 INFO:teuthology.orchestra.run.smithi159.stdout: "channel_device": { 2024-09-19T07:37:19.320 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "channel_device", 2024-09-19T07:37:19.320 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.320 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.321 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.321 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.321 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.321 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.321 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.321 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T07:37:19.321 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.321 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.321 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.321 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.321 INFO:teuthology.orchestra.run.smithi159.stdout: "channel_ident": { 2024-09-19T07:37:19.321 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "channel_ident", 2024-09-19T07:37:19.321 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.322 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.322 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.322 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.322 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.322 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.322 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.322 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T07:37:19.322 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.322 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.322 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.322 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.322 INFO:teuthology.orchestra.run.smithi159.stdout: "channel_perf": { 2024-09-19T07:37:19.322 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "channel_perf", 2024-09-19T07:37:19.322 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.322 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.323 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.323 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.323 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.323 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.323 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.323 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T07:37:19.323 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.323 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.323 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.323 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.323 INFO:teuthology.orchestra.run.smithi159.stdout: "contact": { 2024-09-19T07:37:19.323 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "contact", 2024-09-19T07:37:19.323 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.323 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.324 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.324 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.324 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.324 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.324 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.324 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.324 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.324 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.324 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.324 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.324 INFO:teuthology.orchestra.run.smithi159.stdout: "description": { 2024-09-19T07:37:19.324 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "description", 2024-09-19T07:37:19.324 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.325 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.325 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.325 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.325 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.325 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.325 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.325 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.325 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.325 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.325 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.325 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.325 INFO:teuthology.orchestra.run.smithi159.stdout: "device_url": { 2024-09-19T07:37:19.325 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "device_url", 2024-09-19T07:37:19.325 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.325 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.326 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.326 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T07:37:19.326 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.326 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.326 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.326 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.326 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.326 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.326 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.326 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.326 INFO:teuthology.orchestra.run.smithi159.stdout: "enabled": { 2024-09-19T07:37:19.326 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "enabled", 2024-09-19T07:37:19.326 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.326 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.327 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.327 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.327 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.327 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.327 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.327 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.327 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.327 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.327 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.327 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.327 INFO:teuthology.orchestra.run.smithi159.stdout: "interval": { 2024-09-19T07:37:19.327 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "interval", 2024-09-19T07:37:19.327 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.328 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.328 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.328 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "24", 2024-09-19T07:37:19.328 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "8", 2024-09-19T07:37:19.328 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.328 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.328 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.328 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.328 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.328 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.328 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.328 INFO:teuthology.orchestra.run.smithi159.stdout: "last_opt_revision": { 2024-09-19T07:37:19.328 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "last_opt_revision", 2024-09-19T07:37:19.328 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.329 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.329 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.329 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "1", 2024-09-19T07:37:19.329 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.329 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.329 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.329 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.329 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.329 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.329 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.329 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.329 INFO:teuthology.orchestra.run.smithi159.stdout: "leaderboard": { 2024-09-19T07:37:19.330 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "leaderboard", 2024-09-19T07:37:19.330 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.330 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.330 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.330 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.330 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.330 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.330 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.330 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.330 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.330 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.330 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.330 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.330 INFO:teuthology.orchestra.run.smithi159.stdout: "leaderboard_description": { 2024-09-19T07:37:19.331 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "leaderboard_description", 2024-09-19T07:37:19.331 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.331 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.331 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.331 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.331 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.331 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.331 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.331 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.331 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.331 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.331 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.331 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.332 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.332 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.332 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.332 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.332 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.332 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.332 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.332 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.332 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.332 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.332 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.332 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.332 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.333 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.333 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.333 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.333 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.333 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.333 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.333 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.333 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.333 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.333 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.333 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.333 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.333 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.333 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.334 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.334 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.334 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.334 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.334 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.334 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.334 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.334 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.334 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.334 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.334 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.335 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.335 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.335 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.335 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.335 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.335 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.335 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.335 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.335 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.335 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.335 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.336 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.336 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.336 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.336 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.336 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.336 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.336 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.336 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.336 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.336 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.336 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.336 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.336 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.337 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.337 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.337 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.337 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.337 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.337 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.337 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.337 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.337 INFO:teuthology.orchestra.run.smithi159.stdout: "organization": { 2024-09-19T07:37:19.337 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "organization", 2024-09-19T07:37:19.337 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.337 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.337 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.337 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.338 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.338 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.338 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.338 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.338 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.338 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.338 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.338 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.338 INFO:teuthology.orchestra.run.smithi159.stdout: "proxy": { 2024-09-19T07:37:19.338 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "proxy", 2024-09-19T07:37:19.338 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.338 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.339 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.339 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.339 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.339 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.339 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.339 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.339 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.339 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.339 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.339 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.339 INFO:teuthology.orchestra.run.smithi159.stdout: "url": { 2024-09-19T07:37:19.339 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "url", 2024-09-19T07:37:19.339 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.339 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.340 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.340 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T07:37:19.340 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.340 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.340 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.340 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.340 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.340 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.340 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.340 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.340 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.340 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.340 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.340 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "test_orchestrator", 2024-09-19T07:37:19.341 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.341 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.341 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.341 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.341 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.341 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.341 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.341 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.341 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.341 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.341 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.341 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.341 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.342 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.342 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.342 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.342 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.342 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.342 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.342 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.342 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.342 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.342 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.342 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.342 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.342 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.342 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.343 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.343 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.343 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.343 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.343 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.343 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.343 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.343 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.343 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.343 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.343 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.343 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.343 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.343 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.344 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.344 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.344 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.344 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.344 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.344 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.344 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.344 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.344 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.344 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.344 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.344 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.344 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.344 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.345 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.345 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.345 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.345 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.345 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.345 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.345 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.345 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.345 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.345 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.345 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.345 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.345 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.346 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.346 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.346 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.346 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.346 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.346 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.346 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.346 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.346 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "volumes", 2024-09-19T07:37:19.346 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.346 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.346 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.346 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.346 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.347 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.347 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.347 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.347 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.347 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.347 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.347 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.347 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.347 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.347 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.347 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.347 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.348 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.348 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.348 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.348 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.348 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.348 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.348 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.348 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.348 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.348 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.348 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.348 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.348 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.352 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.352 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.352 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.352 INFO:teuthology.orchestra.run.smithi159.stdout: "max_concurrent_clones": { 2024-09-19T07:37:19.352 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "max_concurrent_clones", 2024-09-19T07:37:19.352 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.352 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.352 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.352 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "4", 2024-09-19T07:37:19.352 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.352 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.352 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.352 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T07:37:19.353 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.353 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.353 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.353 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.353 INFO:teuthology.orchestra.run.smithi159.stdout: "periodic_async_work": { 2024-09-19T07:37:19.353 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "periodic_async_work", 2024-09-19T07:37:19.353 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.353 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.353 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.353 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.353 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.353 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.353 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.354 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Periodically check for async work", 2024-09-19T07:37:19.354 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.354 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.354 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.354 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.354 INFO:teuthology.orchestra.run.smithi159.stdout: "snapshot_clone_delay": { 2024-09-19T07:37:19.354 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "snapshot_clone_delay", 2024-09-19T07:37:19.354 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.354 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.354 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.354 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "0", 2024-09-19T07:37:19.354 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.354 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.354 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.355 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T07:37:19.355 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.355 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.355 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.355 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.355 INFO:teuthology.orchestra.run.smithi159.stdout: "snapshot_clone_no_wait": { 2024-09-19T07:37:19.355 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T07:37:19.355 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.355 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.355 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.355 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "True", 2024-09-19T07:37:19.355 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.355 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.355 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.356 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T07:37:19.356 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.356 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.356 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.356 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.356 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.356 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.356 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.356 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "zabbix", 2024-09-19T07:37:19.356 INFO:teuthology.orchestra.run.smithi159.stdout: "can_run": true, 2024-09-19T07:37:19.356 INFO:teuthology.orchestra.run.smithi159.stdout: "error_string": "", 2024-09-19T07:37:19.356 INFO:teuthology.orchestra.run.smithi159.stdout: "module_options": { 2024-09-19T07:37:19.356 INFO:teuthology.orchestra.run.smithi159.stdout: "discovery_interval": { 2024-09-19T07:37:19.357 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "discovery_interval", 2024-09-19T07:37:19.357 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "uint", 2024-09-19T07:37:19.357 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.357 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.357 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "100", 2024-09-19T07:37:19.357 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.357 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.357 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.357 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.357 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.357 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.357 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.357 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.357 INFO:teuthology.orchestra.run.smithi159.stdout: "identifier": { 2024-09-19T07:37:19.357 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "identifier", 2024-09-19T07:37:19.358 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.358 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.358 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.358 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.358 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.358 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.358 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.358 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.358 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.358 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.358 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.358 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.358 INFO:teuthology.orchestra.run.smithi159.stdout: "interval": { 2024-09-19T07:37:19.358 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "interval", 2024-09-19T07:37:19.359 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "secs", 2024-09-19T07:37:19.359 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.359 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.359 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "60", 2024-09-19T07:37:19.359 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.359 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.359 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.359 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.359 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.359 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.359 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.359 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.359 INFO:teuthology.orchestra.run.smithi159.stdout: "log_level": { 2024-09-19T07:37:19.359 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_level", 2024-09-19T07:37:19.360 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.360 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.360 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.360 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.360 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.360 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.360 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.360 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.360 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.360 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.360 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.360 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.360 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.360 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.361 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.361 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.361 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.361 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.361 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.361 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster": { 2024-09-19T07:37:19.361 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster", 2024-09-19T07:37:19.361 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.361 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.361 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.361 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.361 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.361 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.361 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.361 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.362 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.362 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.362 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.362 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.362 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_cluster_level": { 2024-09-19T07:37:19.362 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_cluster_level", 2024-09-19T07:37:19.362 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.362 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.362 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.362 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "info", 2024-09-19T07:37:19.362 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.362 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.362 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [ 2024-09-19T07:37:19.363 INFO:teuthology.orchestra.run.smithi159.stdout: "", 2024-09-19T07:37:19.363 INFO:teuthology.orchestra.run.smithi159.stdout: "critical", 2024-09-19T07:37:19.363 INFO:teuthology.orchestra.run.smithi159.stdout: "debug", 2024-09-19T07:37:19.363 INFO:teuthology.orchestra.run.smithi159.stdout: "error", 2024-09-19T07:37:19.363 INFO:teuthology.orchestra.run.smithi159.stdout: "info", 2024-09-19T07:37:19.363 INFO:teuthology.orchestra.run.smithi159.stdout: "warning" 2024-09-19T07:37:19.363 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.363 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.363 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.363 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.363 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.363 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.363 INFO:teuthology.orchestra.run.smithi159.stdout: "log_to_file": { 2024-09-19T07:37:19.363 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "log_to_file", 2024-09-19T07:37:19.364 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "bool", 2024-09-19T07:37:19.364 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.364 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 1, 2024-09-19T07:37:19.364 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "False", 2024-09-19T07:37:19.364 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.364 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.364 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.364 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.364 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.364 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.364 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.364 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.364 INFO:teuthology.orchestra.run.smithi159.stdout: "zabbix_host": { 2024-09-19T07:37:19.365 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "zabbix_host", 2024-09-19T07:37:19.365 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.365 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.365 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.365 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "", 2024-09-19T07:37:19.365 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.365 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.365 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.365 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.365 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.365 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.365 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.365 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.366 INFO:teuthology.orchestra.run.smithi159.stdout: "zabbix_port": { 2024-09-19T07:37:19.366 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "zabbix_port", 2024-09-19T07:37:19.366 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "int", 2024-09-19T07:37:19.366 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.366 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.366 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "10051", 2024-09-19T07:37:19.366 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.366 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.366 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.366 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.366 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.366 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.366 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.366 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.367 INFO:teuthology.orchestra.run.smithi159.stdout: "zabbix_sender": { 2024-09-19T07:37:19.367 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "zabbix_sender", 2024-09-19T07:37:19.367 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "str", 2024-09-19T07:37:19.367 INFO:teuthology.orchestra.run.smithi159.stdout: "level": "advanced", 2024-09-19T07:37:19.367 INFO:teuthology.orchestra.run.smithi159.stdout: "flags": 0, 2024-09-19T07:37:19.367 INFO:teuthology.orchestra.run.smithi159.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T07:37:19.367 INFO:teuthology.orchestra.run.smithi159.stdout: "min": "", 2024-09-19T07:37:19.367 INFO:teuthology.orchestra.run.smithi159.stdout: "max": "", 2024-09-19T07:37:19.367 INFO:teuthology.orchestra.run.smithi159.stdout: "enum_allowed": [], 2024-09-19T07:37:19.367 INFO:teuthology.orchestra.run.smithi159.stdout: "desc": "", 2024-09-19T07:37:19.367 INFO:teuthology.orchestra.run.smithi159.stdout: "long_desc": "", 2024-09-19T07:37:19.367 INFO:teuthology.orchestra.run.smithi159.stdout: "tags": [], 2024-09-19T07:37:19.367 INFO:teuthology.orchestra.run.smithi159.stdout: "see_also": [] 2024-09-19T07:37:19.367 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.368 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.368 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.368 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.368 INFO:teuthology.orchestra.run.smithi159.stdout: "services": { 2024-09-19T07:37:19.368 INFO:teuthology.orchestra.run.smithi159.stdout: "dashboard": "https://172.21.15.159:8443/" 2024-09-19T07:37:19.368 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.368 INFO:teuthology.orchestra.run.smithi159.stdout: "always_on_modules": { 2024-09-19T07:37:19.368 INFO:teuthology.orchestra.run.smithi159.stdout: "octopus": [ 2024-09-19T07:37:19.368 INFO:teuthology.orchestra.run.smithi159.stdout: "balancer", 2024-09-19T07:37:19.368 INFO:teuthology.orchestra.run.smithi159.stdout: "crash", 2024-09-19T07:37:19.368 INFO:teuthology.orchestra.run.smithi159.stdout: "devicehealth", 2024-09-19T07:37:19.368 INFO:teuthology.orchestra.run.smithi159.stdout: "orchestrator", 2024-09-19T07:37:19.368 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_autoscaler", 2024-09-19T07:37:19.369 INFO:teuthology.orchestra.run.smithi159.stdout: "progress", 2024-09-19T07:37:19.369 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_support", 2024-09-19T07:37:19.369 INFO:teuthology.orchestra.run.smithi159.stdout: "status", 2024-09-19T07:37:19.369 INFO:teuthology.orchestra.run.smithi159.stdout: "telemetry", 2024-09-19T07:37:19.369 INFO:teuthology.orchestra.run.smithi159.stdout: "volumes" 2024-09-19T07:37:19.369 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.369 INFO:teuthology.orchestra.run.smithi159.stdout: "pacific": [ 2024-09-19T07:37:19.369 INFO:teuthology.orchestra.run.smithi159.stdout: "balancer", 2024-09-19T07:37:19.369 INFO:teuthology.orchestra.run.smithi159.stdout: "crash", 2024-09-19T07:37:19.369 INFO:teuthology.orchestra.run.smithi159.stdout: "devicehealth", 2024-09-19T07:37:19.369 INFO:teuthology.orchestra.run.smithi159.stdout: "orchestrator", 2024-09-19T07:37:19.369 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_autoscaler", 2024-09-19T07:37:19.369 INFO:teuthology.orchestra.run.smithi159.stdout: "progress", 2024-09-19T07:37:19.369 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_support", 2024-09-19T07:37:19.370 INFO:teuthology.orchestra.run.smithi159.stdout: "status", 2024-09-19T07:37:19.370 INFO:teuthology.orchestra.run.smithi159.stdout: "telemetry", 2024-09-19T07:37:19.370 INFO:teuthology.orchestra.run.smithi159.stdout: "volumes" 2024-09-19T07:37:19.370 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.370 INFO:teuthology.orchestra.run.smithi159.stdout: "quincy": [ 2024-09-19T07:37:19.370 INFO:teuthology.orchestra.run.smithi159.stdout: "balancer", 2024-09-19T07:37:19.370 INFO:teuthology.orchestra.run.smithi159.stdout: "crash", 2024-09-19T07:37:19.370 INFO:teuthology.orchestra.run.smithi159.stdout: "devicehealth", 2024-09-19T07:37:19.370 INFO:teuthology.orchestra.run.smithi159.stdout: "orchestrator", 2024-09-19T07:37:19.370 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_autoscaler", 2024-09-19T07:37:19.370 INFO:teuthology.orchestra.run.smithi159.stdout: "progress", 2024-09-19T07:37:19.370 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_support", 2024-09-19T07:37:19.370 INFO:teuthology.orchestra.run.smithi159.stdout: "status", 2024-09-19T07:37:19.370 INFO:teuthology.orchestra.run.smithi159.stdout: "telemetry", 2024-09-19T07:37:19.371 INFO:teuthology.orchestra.run.smithi159.stdout: "volumes" 2024-09-19T07:37:19.371 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.371 INFO:teuthology.orchestra.run.smithi159.stdout: "reef": [ 2024-09-19T07:37:19.371 INFO:teuthology.orchestra.run.smithi159.stdout: "balancer", 2024-09-19T07:37:19.371 INFO:teuthology.orchestra.run.smithi159.stdout: "crash", 2024-09-19T07:37:19.371 INFO:teuthology.orchestra.run.smithi159.stdout: "devicehealth", 2024-09-19T07:37:19.371 INFO:teuthology.orchestra.run.smithi159.stdout: "orchestrator", 2024-09-19T07:37:19.371 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_autoscaler", 2024-09-19T07:37:19.371 INFO:teuthology.orchestra.run.smithi159.stdout: "progress", 2024-09-19T07:37:19.371 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_support", 2024-09-19T07:37:19.371 INFO:teuthology.orchestra.run.smithi159.stdout: "status", 2024-09-19T07:37:19.371 INFO:teuthology.orchestra.run.smithi159.stdout: "telemetry", 2024-09-19T07:37:19.371 INFO:teuthology.orchestra.run.smithi159.stdout: "volumes" 2024-09-19T07:37:19.371 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-09-19T07:37:19.372 INFO:teuthology.orchestra.run.smithi159.stdout: "squid": [ 2024-09-19T07:37:19.372 INFO:teuthology.orchestra.run.smithi159.stdout: "balancer", 2024-09-19T07:37:19.372 INFO:teuthology.orchestra.run.smithi159.stdout: "crash", 2024-09-19T07:37:19.372 INFO:teuthology.orchestra.run.smithi159.stdout: "devicehealth", 2024-09-19T07:37:19.372 INFO:teuthology.orchestra.run.smithi159.stdout: "orchestrator", 2024-09-19T07:37:19.372 INFO:teuthology.orchestra.run.smithi159.stdout: "pg_autoscaler", 2024-09-19T07:37:19.372 INFO:teuthology.orchestra.run.smithi159.stdout: "progress", 2024-09-19T07:37:19.372 INFO:teuthology.orchestra.run.smithi159.stdout: "rbd_support", 2024-09-19T07:37:19.372 INFO:teuthology.orchestra.run.smithi159.stdout: "status", 2024-09-19T07:37:19.372 INFO:teuthology.orchestra.run.smithi159.stdout: "telemetry", 2024-09-19T07:37:19.372 INFO:teuthology.orchestra.run.smithi159.stdout: "volumes" 2024-09-19T07:37:19.372 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-19T07:37:19.372 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.372 INFO:teuthology.orchestra.run.smithi159.stdout: "last_failure_osd_epoch": 3, 2024-09-19T07:37:19.373 INFO:teuthology.orchestra.run.smithi159.stdout: "active_clients": [ 2024-09-19T07:37:19.373 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.373 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "devicehealth", 2024-09-19T07:37:19.373 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-19T07:37:19.373 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.373 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-19T07:37:19.373 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:0", 2024-09-19T07:37:19.373 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 573283438 2024-09-19T07:37:19.373 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.373 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-19T07:37:19.373 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.373 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.373 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "libcephsqlite", 2024-09-19T07:37:19.373 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-19T07:37:19.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.374 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-19T07:37:19.374 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:0", 2024-09-19T07:37:19.374 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2691587722 2024-09-19T07:37:19.374 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.374 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-19T07:37:19.374 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.374 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "rbd_support", 2024-09-19T07:37:19.374 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-19T07:37:19.374 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.374 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-19T07:37:19.374 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:0", 2024-09-19T07:37:19.374 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 2218622084 2024-09-19T07:37:19.375 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.375 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-19T07:37:19.375 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:19.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.375 INFO:teuthology.orchestra.run.smithi159.stdout: "name": "volumes", 2024-09-19T07:37:19.375 INFO:teuthology.orchestra.run.smithi159.stdout: "addrvec": [ 2024-09-19T07:37:19.375 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-09-19T07:37:19.375 INFO:teuthology.orchestra.run.smithi159.stdout: "type": "v2", 2024-09-19T07:37:19.375 INFO:teuthology.orchestra.run.smithi159.stdout: "addr": "172.21.15.159:0", 2024-09-19T07:37:19.375 INFO:teuthology.orchestra.run.smithi159.stdout: "nonce": 912568592 2024-09-19T07:37:19.375 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.375 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-19T07:37:19.375 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:19.375 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-19T07:37:19.375 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-19T07:37:19.377 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:17 smithi189 ceph-mon[34851]: pgmap v503: 129 pgs: 129 active+clean; 4.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 120 op/s 2024-09-19T07:37:19.377 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:17 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/172004166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T07:37:19.377 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:17 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3053366834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-19T07:37:19.378 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:18 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/340850915' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T07:37:19.378 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:17 smithi203 ceph-mon[35335]: pgmap v503: 129 pgs: 129 active+clean; 4.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 120 op/s 2024-09-19T07:37:19.378 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:17 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/172004166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T07:37:19.378 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:17 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3053366834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-19T07:37:19.378 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:18 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/340850915' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T07:37:19.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:18.029+0000 7fa96e800640 1 -- 172.21.15.159:0/340850915 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa94c0a3250 msgr2=0x7fa94c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:19.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:18.029+0000 7fa96e800640 1 --2- 172.21.15.159:0/340850915 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa94c0a3250 0x7fa94c0a5710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa9580314e0 tx=0x7fa958000bc0 comp rx=0 tx=0).stop 2024-09-19T07:37:19.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:18.029+0000 7fa96e800640 1 -- 172.21.15.159:0/340850915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa968077df0 msgr2=0x7fa96819e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:19.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:18.029+0000 7fa96e800640 1 --2- 172.21.15.159:0/340850915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa968077df0 0x7fa96819e9c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa95400ea60 tx=0x7fa95400ee60 comp rx=0 tx=0).stop 2024-09-19T07:37:19.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:18.029+0000 7fa96e800640 1 -- 172.21.15.159:0/340850915 shutdown_connections 2024-09-19T07:37:19.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:18.029+0000 7fa96e800640 1 --2- 172.21.15.159:0/340850915 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa94c0a3250 0x7fa94c0a5710 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:19.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:18.029+0000 7fa96e800640 1 --2- 172.21.15.159:0/340850915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa968078790 0x7fa9681a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:19.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:18.029+0000 7fa96e800640 1 --2- 172.21.15.159:0/340850915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa968077df0 0x7fa96819e9c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:19.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:18.029+0000 7fa96e800640 1 --2- 172.21.15.159:0/340850915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa968076bb0 0x7fa96819e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:19.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:18.029+0000 7fa96e800640 1 -- 172.21.15.159:0/340850915 >> 172.21.15.159:0/340850915 conn(0x7fa9680ff4b0 msgr2=0x7fa968100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:19.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:18.029+0000 7fa96e800640 1 -- 172.21.15.159:0/340850915 shutdown_connections 2024-09-19T07:37:19.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:18.029+0000 7fa96e800640 1 -- 172.21.15.159:0/340850915 wait complete. 2024-09-19T07:37:19.385 DEBUG:tasks.check_counter:Getting stats from a 2024-09-19T07:37:19.385 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-19T07:37:19.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:17 smithi159 ceph-mon[30472]: pgmap v503: 129 pgs: 129 active+clean; 4.7 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 120 op/s 2024-09-19T07:37:19.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:17 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/172004166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T07:37:19.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:17 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3053366834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-19T07:37:19.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:18 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/340850915' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T07:37:19.763 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:20.271 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:19 smithi159 ceph-mon[30472]: pgmap v504: 129 pgs: 129 active+clean; 1.9 GiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 179 op/s 2024-09-19T07:37:20.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:19 smithi203 ceph-mon[35335]: pgmap v504: 129 pgs: 129 active+clean; 1.9 GiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 179 op/s 2024-09-19T07:37:20.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:19 smithi189 ceph-mon[34851]: pgmap v504: 129 pgs: 129 active+clean; 1.9 GiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 179 op/s 2024-09-19T07:37:20.416 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-19T07:37:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:37:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 0, 2024-09-19T07:37:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 2, 2024-09-19T07:37:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 0, 2024-09-19T07:37:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 0, 2024-09-19T07:37:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 4, 2024-09-19T07:37:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 0, 2024-09-19T07:37:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.003777463, 2024-09-19T07:37:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T07:37:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.000822015, 2024-09-19T07:37:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:20.417 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:37:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:37:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:37:20.418 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1, 2024-09-19T07:37:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 82, 2024-09-19T07:37:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 160, 2024-09-19T07:37:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 92960, 2024-09-19T07:37:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 3, 2024-09-19T07:37:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2024-09-19T07:37:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.010827928, 2024-09-19T07:37:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.007664882, 2024-09-19T07:37:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.001926639, 2024-09-19T07:37:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 80, 2024-09-19T07:37:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.009186978, 2024-09-19T07:37:20.419 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000114837 2024-09-19T07:37:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T07:37:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 92960 2024-09-19T07:37:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:37:20.420 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 115, 2024-09-19T07:37:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 110, 2024-09-19T07:37:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 451968, 2024-09-19T07:37:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 66256, 2024-09-19T07:37:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 2, 2024-09-19T07:37:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 1, 2024-09-19T07:37:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.026214965, 2024-09-19T07:37:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.011178848, 2024-09-19T07:37:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.019413877, 2024-09-19T07:37:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:20.421 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 110, 2024-09-19T07:37:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.017675947, 2024-09-19T07:37:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000160690 2024-09-19T07:37:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 451968, 2024-09-19T07:37:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 66256 2024-09-19T07:37:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-09-19T07:37:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 0, 2024-09-19T07:37:20.422 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-09-19T07:37:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-09-19T07:37:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:37:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-09-19T07:37:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:37:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:37:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:37:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:37:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:37:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:37:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:37:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:37:20.423 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:37:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:37:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:37:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:37:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:37:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-09-19T07:37:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:37:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-09-19T07:37:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:37:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:37:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:37:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:37:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:37:20.424 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:37:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:37:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-09-19T07:37:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:37:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:37:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:37:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-09-19T07:37:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-09-19T07:37:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-09-19T07:37:20.425 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:37:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:37:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:37:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:37:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 149854, 2024-09-19T07:37:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 28, 2024-09-19T07:37:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:37:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-09-19T07:37:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-09-19T07:37:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-09-19T07:37:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:37:20.426 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-09-19T07:37:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:37:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-09-19T07:37:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 0, 2024-09-19T07:37:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 0, 2024-09-19T07:37:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:37:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:37:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-09-19T07:37:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-09-19T07:37:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 928, 2024-09-19T07:37:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 2, 2024-09-19T07:37:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-09-19T07:37:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-09-19T07:37:20.427 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-09-19T07:37:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-09-19T07:37:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:37:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-19T07:37:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-19T07:37:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-19T07:37:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 112, 2024-09-19T07:37:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 433291, 2024-09-19T07:37:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 112, 2024-09-19T07:37:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-19T07:37:20.428 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-19T07:37:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 112, 2024-09-19T07:37:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 433291, 2024-09-19T07:37:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-19T07:37:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:20.429 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:20.429 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-19T07:37:21.371 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-19T07:37:21.372 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-19T07:37:21.372 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-19T07:37:21.372 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-19T07:37:21.372 DEBUG:tasks.check_counter:Getting stats from d 2024-09-19T07:37:21.372 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-19T07:37:21.745 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:22.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:21 smithi159 ceph-mon[30472]: pgmap v505: 129 pgs: 129 active+clean; 1.2 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 220 op/s 2024-09-19T07:37:22.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:21 smithi203 ceph-mon[35335]: pgmap v505: 129 pgs: 129 active+clean; 1.2 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 220 op/s 2024-09-19T07:37:22.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:21 smithi189 ceph-mon[34851]: pgmap v505: 129 pgs: 129 active+clean; 1.2 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 220 op/s 2024-09-19T07:37:22.426 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-19T07:37:22.426 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:37:22.426 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 0, 2024-09-19T07:37:22.426 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 2, 2024-09-19T07:37:22.426 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 0, 2024-09-19T07:37:22.427 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 0, 2024-09-19T07:37:22.427 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 4, 2024-09-19T07:37:22.427 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 0, 2024-09-19T07:37:22.427 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.003410382, 2024-09-19T07:37:22.427 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T07:37:22.427 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.000493965, 2024-09-19T07:37:22.427 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:22.427 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:22.427 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:22.427 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:22.427 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:22.427 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:22.427 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:22.428 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:22.428 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:22.428 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:22.428 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:22.428 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:37:22.428 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:37:22.428 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:22.428 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:37:22.428 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1, 2024-09-19T07:37:22.428 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 87, 2024-09-19T07:37:22.428 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 160, 2024-09-19T07:37:22.428 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 93840, 2024-09-19T07:37:22.429 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 3, 2024-09-19T07:37:22.429 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2024-09-19T07:37:22.429 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.011087258, 2024-09-19T07:37:22.429 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.008385461, 2024-09-19T07:37:22.429 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.001409810, 2024-09-19T07:37:22.429 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:22.429 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:22.429 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 85, 2024-09-19T07:37:22.429 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.024408110, 2024-09-19T07:37:22.429 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000287154 2024-09-19T07:37:22.429 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:22.429 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:22.430 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:22.430 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:22.430 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:22.430 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:22.430 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T07:37:22.430 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 93840 2024-09-19T07:37:22.430 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:22.430 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:37:22.430 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 122, 2024-09-19T07:37:22.430 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 117, 2024-09-19T07:37:22.430 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 453936, 2024-09-19T07:37:22.430 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 70256, 2024-09-19T07:37:22.430 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 2, 2024-09-19T07:37:22.430 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 1, 2024-09-19T07:37:22.431 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.028844705, 2024-09-19T07:37:22.431 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.012750249, 2024-09-19T07:37:22.431 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.020283295, 2024-09-19T07:37:22.431 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:22.431 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:22.431 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 117, 2024-09-19T07:37:22.431 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.018919307, 2024-09-19T07:37:22.431 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000161703 2024-09-19T07:37:22.431 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:22.431 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:22.431 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:22.431 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:22.431 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:22.432 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:22.432 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 453936, 2024-09-19T07:37:22.432 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 70256 2024-09-19T07:37:22.432 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:22.432 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-09-19T07:37:22.432 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 0, 2024-09-19T07:37:22.432 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-09-19T07:37:22.432 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:22.432 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-09-19T07:37:22.432 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:37:22.432 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-09-19T07:37:22.432 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:37:22.433 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:37:22.433 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:37:22.433 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:37:22.433 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:37:22.433 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:37:22.433 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:37:22.433 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:37:22.433 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:37:22.433 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:37:22.433 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:37:22.433 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:37:22.433 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:37:22.433 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-09-19T07:37:22.433 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:37:22.434 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-09-19T07:37:22.434 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:37:22.434 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:37:22.434 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:37:22.434 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:37:22.434 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:37:22.434 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:37:22.434 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:37:22.434 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-09-19T07:37:22.434 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:37:22.434 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:37:22.434 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:37:22.435 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-09-19T07:37:22.435 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-09-19T07:37:22.435 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-09-19T07:37:22.435 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:37:22.435 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:37:22.435 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:37:22.435 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:37:22.435 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 148823, 2024-09-19T07:37:22.435 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 27, 2024-09-19T07:37:22.435 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:37:22.435 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-09-19T07:37:22.435 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-09-19T07:37:22.435 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-09-19T07:37:22.436 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:37:22.436 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-09-19T07:37:22.436 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:37:22.436 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-09-19T07:37:22.436 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 0, 2024-09-19T07:37:22.436 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 0, 2024-09-19T07:37:22.436 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:37:22.436 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:37:22.436 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-09-19T07:37:22.436 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-09-19T07:37:22.436 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 928, 2024-09-19T07:37:22.436 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 2, 2024-09-19T07:37:22.436 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-09-19T07:37:22.437 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-09-19T07:37:22.437 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-09-19T07:37:22.437 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-09-19T07:37:22.437 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:22.437 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:37:22.437 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-19T07:37:22.437 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-19T07:37:22.437 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-19T07:37:22.437 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 119, 2024-09-19T07:37:22.437 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 434265, 2024-09-19T07:37:22.437 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:22.437 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 119, 2024-09-19T07:37:22.437 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-19T07:37:22.437 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-19T07:37:22.438 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 119, 2024-09-19T07:37:22.438 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 434265, 2024-09-19T07:37:22.438 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-19T07:37:22.438 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:22.438 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:22.438 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:22.438 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:22.438 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:22.438 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-19T07:37:23.404 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-19T07:37:23.404 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-19T07:37:23.404 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-19T07:37:23.404 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-19T07:37:23.404 DEBUG:tasks.check_counter:Getting stats from g 2024-09-19T07:37:23.405 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-19T07:37:23.782 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:24.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:23 smithi203 ceph-mon[35335]: pgmap v506: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.7 KiB/s wr, 222 op/s 2024-09-19T07:37:24.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:23 smithi189 ceph-mon[34851]: pgmap v506: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.7 KiB/s wr, 222 op/s 2024-09-19T07:37:24.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:23 smithi159 ceph-mon[30472]: pgmap v506: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.7 KiB/s wr, 222 op/s 2024-09-19T07:37:24.966 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-19T07:37:24.966 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:37:24.966 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 113, 2024-09-19T07:37:24.966 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 110, 2024-09-19T07:37:24.966 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 449488, 2024-09-19T07:37:24.966 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 67088, 2024-09-19T07:37:24.966 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 4, 2024-09-19T07:37:24.966 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 0, 2024-09-19T07:37:24.967 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.027003310, 2024-09-19T07:37:24.967 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.011690193, 2024-09-19T07:37:24.967 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.017021643, 2024-09-19T07:37:24.967 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:24.967 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:24.967 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 109, 2024-09-19T07:37:24.967 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.014115119, 2024-09-19T07:37:24.967 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000129496 2024-09-19T07:37:24.967 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:24.967 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:24.967 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:24.967 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:24.968 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:24.968 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:24.968 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 449488, 2024-09-19T07:37:24.968 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 67088 2024-09-19T07:37:24.968 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:24.968 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:37:24.968 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 5, 2024-09-19T07:37:24.968 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 86, 2024-09-19T07:37:24.968 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 3184, 2024-09-19T07:37:24.968 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 93760, 2024-09-19T07:37:24.968 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 3, 2024-09-19T07:37:24.968 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2024-09-19T07:37:24.969 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.012846350, 2024-09-19T07:37:24.969 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.007198712, 2024-09-19T07:37:24.969 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.004498958, 2024-09-19T07:37:24.969 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:24.969 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:24.969 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 85, 2024-09-19T07:37:24.969 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.013557527, 2024-09-19T07:37:24.969 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000159500 2024-09-19T07:37:24.969 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:24.969 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:24.969 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:24.969 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:24.969 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:24.970 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:24.970 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-19T07:37:24.970 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 93760 2024-09-19T07:37:24.970 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:24.970 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:37:24.970 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 0, 2024-09-19T07:37:24.970 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 2, 2024-09-19T07:37:24.970 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 0, 2024-09-19T07:37:24.970 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 0, 2024-09-19T07:37:24.970 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 2, 2024-09-19T07:37:24.970 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 1, 2024-09-19T07:37:24.970 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.000644036, 2024-09-19T07:37:24.970 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T07:37:24.970 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.000545227, 2024-09-19T07:37:24.971 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:24.971 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:24.971 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:24.971 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:24.971 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:24.971 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:24.971 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:24.971 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:24.971 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:24.971 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:24.971 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:24.971 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:37:24.972 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:37:24.972 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:24.972 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-09-19T07:37:24.972 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 0, 2024-09-19T07:37:24.972 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-09-19T07:37:24.972 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:24.972 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-09-19T07:37:24.972 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:37:24.972 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-09-19T07:37:24.972 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:37:24.972 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:37:24.972 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:37:24.972 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:37:24.973 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:37:24.973 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:37:24.973 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:37:24.973 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:37:24.973 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:37:24.973 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:37:24.973 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:37:24.973 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:37:24.973 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:37:24.973 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-09-19T07:37:24.973 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:37:24.973 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-09-19T07:37:24.973 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:37:24.973 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:37:24.974 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:37:24.974 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:37:24.974 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:37:24.974 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:37:24.974 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:37:24.974 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-09-19T07:37:24.974 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:37:24.974 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:37:24.974 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:37:24.974 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-09-19T07:37:24.974 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-09-19T07:37:24.974 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-09-19T07:37:24.974 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:37:24.975 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:37:24.975 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:37:24.975 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:37:24.975 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 148823, 2024-09-19T07:37:24.975 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 27, 2024-09-19T07:37:24.975 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:37:24.975 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-09-19T07:37:24.975 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-09-19T07:37:24.975 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-09-19T07:37:24.975 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:37:24.975 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-09-19T07:37:24.975 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:37:24.975 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-09-19T07:37:24.975 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 0, 2024-09-19T07:37:24.976 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 0, 2024-09-19T07:37:24.976 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:37:24.976 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:37:24.976 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-09-19T07:37:24.976 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-09-19T07:37:24.976 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 928, 2024-09-19T07:37:24.976 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 2, 2024-09-19T07:37:24.976 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-09-19T07:37:24.976 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-09-19T07:37:24.976 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-09-19T07:37:24.976 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-09-19T07:37:24.976 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:24.976 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:37:24.976 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-19T07:37:24.977 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-19T07:37:24.977 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-19T07:37:24.977 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 114, 2024-09-19T07:37:24.977 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 433551, 2024-09-19T07:37:24.977 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:24.977 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 114, 2024-09-19T07:37:24.977 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-19T07:37:24.977 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-19T07:37:24.977 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 114, 2024-09-19T07:37:24.977 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 433551, 2024-09-19T07:37:24.977 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-19T07:37:24.977 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:24.977 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:24.977 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:24.977 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:24.978 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:24.978 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-19T07:37:25.799 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-19T07:37:25.800 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-19T07:37:25.800 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-19T07:37:25.800 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-19T07:37:25.800 DEBUG:tasks.check_counter:Getting stats from j 2024-09-19T07:37:25.800 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-19T07:37:26.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:25 smithi159 ceph-mon[30472]: pgmap v507: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 219 op/s 2024-09-19T07:37:26.173 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:26.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:25 smithi203 ceph-mon[35335]: pgmap v507: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 219 op/s 2024-09-19T07:37:26.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:25 smithi189 ceph-mon[34851]: pgmap v507: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 219 op/s 2024-09-19T07:37:26.829 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-19T07:37:26.829 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:37:26.829 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 4, 2024-09-19T07:37:26.829 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 4, 2024-09-19T07:37:26.829 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 3024, 2024-09-19T07:37:26.829 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 448, 2024-09-19T07:37:26.829 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 4, 2024-09-19T07:37:26.830 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 0, 2024-09-19T07:37:26.830 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.006184156, 2024-09-19T07:37:26.830 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.000058668, 2024-09-19T07:37:26.830 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.003461116, 2024-09-19T07:37:26.830 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:26.830 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:26.830 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3, 2024-09-19T07:37:26.830 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.006029567, 2024-09-19T07:37:26.830 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002009855 2024-09-19T07:37:26.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:26.830 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:26.830 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:26.830 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:26.831 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:26.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:26.831 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T07:37:26.831 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T07:37:26.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:26.831 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:37:26.831 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1, 2024-09-19T07:37:26.831 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 89, 2024-09-19T07:37:26.831 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 160, 2024-09-19T07:37:26.831 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 94192, 2024-09-19T07:37:26.831 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 3, 2024-09-19T07:37:26.831 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2024-09-19T07:37:26.831 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.011168988, 2024-09-19T07:37:26.832 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.008300024, 2024-09-19T07:37:26.832 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.001552176, 2024-09-19T07:37:26.832 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:26.832 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:26.832 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 87, 2024-09-19T07:37:26.832 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.007542095, 2024-09-19T07:37:26.832 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000086690 2024-09-19T07:37:26.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:26.832 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:26.832 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:26.832 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:26.832 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:26.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:26.833 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T07:37:26.833 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 94192 2024-09-19T07:37:26.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:26.833 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:37:26.833 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 120, 2024-09-19T07:37:26.833 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 117, 2024-09-19T07:37:26.833 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 451456, 2024-09-19T07:37:26.833 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 71088, 2024-09-19T07:37:26.833 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 2, 2024-09-19T07:37:26.833 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 1, 2024-09-19T07:37:26.833 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.028646544, 2024-09-19T07:37:26.833 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.014365186, 2024-09-19T07:37:26.834 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.019038143, 2024-09-19T07:37:26.834 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:26.834 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:26.834 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 116, 2024-09-19T07:37:26.834 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.012076667, 2024-09-19T07:37:26.834 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000104109 2024-09-19T07:37:26.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:26.834 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:26.834 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:26.834 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:26.834 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:26.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:26.834 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 451456, 2024-09-19T07:37:26.835 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 71088 2024-09-19T07:37:26.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:26.835 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-09-19T07:37:26.835 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 0, 2024-09-19T07:37:26.835 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-09-19T07:37:26.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:26.835 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-09-19T07:37:26.835 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:37:26.835 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-09-19T07:37:26.835 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:37:26.835 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:37:26.835 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:37:26.836 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:37:26.836 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:37:26.836 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:37:26.836 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:37:26.836 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:37:26.836 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:37:26.836 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:37:26.836 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:37:26.836 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:37:26.836 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:37:26.836 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-09-19T07:37:26.837 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:37:26.837 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-09-19T07:37:26.837 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:37:26.837 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:37:26.837 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:37:26.837 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:37:26.837 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:37:26.837 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:37:26.837 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:37:26.837 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-09-19T07:37:26.837 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:37:26.837 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:37:26.838 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:37:26.838 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-09-19T07:37:26.838 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-09-19T07:37:26.838 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-09-19T07:37:26.838 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:37:26.838 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:37:26.838 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:37:26.838 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:37:26.838 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 149854, 2024-09-19T07:37:26.838 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 28, 2024-09-19T07:37:26.838 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:37:26.838 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-09-19T07:37:26.839 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-09-19T07:37:26.839 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-09-19T07:37:26.839 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:37:26.839 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-09-19T07:37:26.839 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:37:26.839 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-09-19T07:37:26.839 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 0, 2024-09-19T07:37:26.839 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 0, 2024-09-19T07:37:26.839 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:37:26.839 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:37:26.839 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-09-19T07:37:26.839 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-09-19T07:37:26.839 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 928, 2024-09-19T07:37:26.839 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 2, 2024-09-19T07:37:26.840 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-09-19T07:37:26.840 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-09-19T07:37:26.840 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-09-19T07:37:26.840 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-09-19T07:37:26.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-19T07:37:26.840 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:37:26.840 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-19T07:37:26.840 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-19T07:37:26.840 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-19T07:37:26.840 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 121, 2024-09-19T07:37:26.840 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 434525, 2024-09-19T07:37:26.840 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:26.840 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 121, 2024-09-19T07:37:26.840 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-19T07:37:26.841 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-19T07:37:26.841 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 121, 2024-09-19T07:37:26.841 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 434525, 2024-09-19T07:37:26.841 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-19T07:37:26.841 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-19T07:37:26.841 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-19T07:37:26.841 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-19T07:37:26.841 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:26.841 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-19T07:37:26.841 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-19T07:37:27.858 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-19T07:37:27.858 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-19T07:37:27.858 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-19T07:37:27.858 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-19T07:37:27.858 DEBUG:tasks.check_counter:Getting stats from b 2024-09-19T07:37:27.858 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-19T07:37:28.118 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:37:28.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:27 smithi159 ceph-mon[30472]: pgmap v508: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 210 op/s 2024-09-19T07:37:28.239 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:27 smithi189 ceph-mon[34851]: pgmap v508: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 210 op/s 2024-09-19T07:37:28.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:27 smithi203 ceph-mon[35335]: pgmap v508: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 210 op/s 2024-09-19T07:37:28.706 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-19T07:37:28.706 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:37:28.707 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1696, 2024-09-19T07:37:28.707 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1616, 2024-09-19T07:37:28.707 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 888439, 2024-09-19T07:37:28.707 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 549508, 2024-09-19T07:37:28.707 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 8, 2024-09-19T07:37:28.707 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-09-19T07:37:28.707 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.118657332, 2024-09-19T07:37:28.707 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.039749019, 2024-09-19T07:37:28.707 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.054034112, 2024-09-19T07:37:28.707 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.029070627, 2024-09-19T07:37:28.707 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:28.707 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1615, 2024-09-19T07:37:28.707 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.313882873, 2024-09-19T07:37:28.708 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000194354 2024-09-19T07:37:28.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.708 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:28.708 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 359, 2024-09-19T07:37:28.708 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000068631, 2024-09-19T07:37:28.708 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000191 2024-09-19T07:37:28.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.708 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 500976, 2024-09-19T07:37:28.708 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 75296 2024-09-19T07:37:28.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.708 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:37:28.708 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1956, 2024-09-19T07:37:28.708 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1841, 2024-09-19T07:37:28.709 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 567351, 2024-09-19T07:37:28.709 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 845919, 2024-09-19T07:37:28.709 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 12, 2024-09-19T07:37:28.709 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 6, 2024-09-19T07:37:28.709 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.150877981, 2024-09-19T07:37:28.709 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.040982633, 2024-09-19T07:37:28.709 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.054349233, 2024-09-19T07:37:28.709 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.051101860, 2024-09-19T07:37:28.709 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:28.709 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1840, 2024-09-19T07:37:28.709 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.278997642, 2024-09-19T07:37:28.709 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000151629 2024-09-19T07:37:28.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.709 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:28.710 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 407, 2024-09-19T07:37:28.710 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000167693, 2024-09-19T07:37:28.710 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000412 2024-09-19T07:37:28.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.710 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 62736, 2024-09-19T07:37:28.710 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 258304 2024-09-19T07:37:28.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.710 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:37:28.710 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 2919, 2024-09-19T07:37:28.710 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 2846, 2024-09-19T07:37:28.710 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 778346, 2024-09-19T07:37:28.710 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 1653728, 2024-09-19T07:37:28.710 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 13, 2024-09-19T07:37:28.710 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 6, 2024-09-19T07:37:28.711 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.198055647, 2024-09-19T07:37:28.711 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.062052840, 2024-09-19T07:37:28.711 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.063411600, 2024-09-19T07:37:28.711 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.065848311, 2024-09-19T07:37:28.711 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:28.711 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2844, 2024-09-19T07:37:28.711 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.347070936, 2024-09-19T07:37:28.711 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000122036 2024-09-19T07:37:28.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.711 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:28.711 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 405, 2024-09-19T07:37:28.711 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000100384, 2024-09-19T07:37:28.711 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000247 2024-09-19T07:37:28.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.712 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:37:28.712 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:37:28.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.712 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-09-19T07:37:28.712 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-09-19T07:37:28.712 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-09-19T07:37:28.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.712 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-09-19T07:37:28.712 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-09-19T07:37:28.712 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-09-19T07:37:28.712 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 445, 2024-09-19T07:37:28.712 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.085573586, 2024-09-19T07:37:28.712 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000192300 2024-09-19T07:37:28.713 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:28.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.713 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-09-19T07:37:28.713 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-09-19T07:37:28.713 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-09-19T07:37:28.713 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 4441, 2024-09-19T07:37:28.713 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.119621306, 2024-09-19T07:37:28.713 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000026935 2024-09-19T07:37:28.713 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:28.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.713 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-09-19T07:37:28.713 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 64, 2024-09-19T07:37:28.713 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 64, 2024-09-19T07:37:28.713 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-09-19T07:37:28.713 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 64, 2024-09-19T07:37:28.714 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.063398854, 2024-09-19T07:37:28.714 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000990607 2024-09-19T07:37:28.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.714 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-09-19T07:37:28.714 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 0, 2024-09-19T07:37:28.714 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-09-19T07:37:28.714 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-09-19T07:37:28.714 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 12, 2024-09-19T07:37:28.714 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-09-19T07:37:28.714 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-09-19T07:37:28.714 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 25, 2024-09-19T07:37:28.714 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 0, 2024-09-19T07:37:28.714 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 25, 2024-09-19T07:37:28.714 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-09-19T07:37:28.714 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 24, 2024-09-19T07:37:28.715 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 0, 2024-09-19T07:37:28.715 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 2, 2024-09-19T07:37:28.715 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 2, 2024-09-19T07:37:28.715 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 2, 2024-09-19T07:37:28.715 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 90, 2024-09-19T07:37:28.715 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 83, 2024-09-19T07:37:28.715 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-09-19T07:37:28.715 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 0, 2024-09-19T07:37:28.715 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:37:28.715 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-09-19T07:37:28.716 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-09-19T07:37:28.716 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 126, 2024-09-19T07:37:28.716 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-09-19T07:37:28.716 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-09-19T07:37:28.716 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-09-19T07:37:28.716 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-09-19T07:37:28.716 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-09-19T07:37:28.716 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-09-19T07:37:28.716 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:37:28.716 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-09-19T07:37:28.716 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 4, 2024-09-19T07:37:28.716 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 234, 2024-09-19T07:37:28.716 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 0, 2024-09-19T07:37:28.717 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:37:28.717 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-09-19T07:37:28.717 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:37:28.717 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:37:28.717 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:37:28.717 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:37:28.717 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:37:28.718 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-09-19T07:37:28.718 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:37:28.718 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 12, 2024-09-19T07:37:28.718 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 108, 2024-09-19T07:37:28.718 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:37:28.718 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:37:28.718 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:37:28.718 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 153, 2024-09-19T07:37:28.718 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 134, 2024-09-19T07:37:28.718 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 5, 2024-09-19T07:37:28.718 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 25 2024-09-19T07:37:28.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.718 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-09-19T07:37:28.718 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 5, 2024-09-19T07:37:28.719 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-09-19T07:37:28.719 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-09-19T07:37:28.719 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 0, 2024-09-19T07:37:28.719 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 12, 2024-09-19T07:37:28.719 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-09-19T07:37:28.719 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-09-19T07:37:28.719 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:37:28.719 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 5, 2024-09-19T07:37:28.719 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 5, 2024-09-19T07:37:28.719 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-09-19T07:37:28.719 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-09-19T07:37:28.719 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-09-19T07:37:28.720 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:37:28.720 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:37:28.720 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-09-19T07:37:28.720 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-09-19T07:37:28.720 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 0, 2024-09-19T07:37:28.720 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 0, 2024-09-19T07:37:28.720 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:37:28.720 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-09-19T07:37:28.720 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-09-19T07:37:28.721 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:37:28.721 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-09-19T07:37:28.721 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-09-19T07:37:28.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.721 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-09-19T07:37:28.722 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-09-19T07:37:28.722 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 202, 2024-09-19T07:37:28.722 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-09-19T07:37:28.722 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-09-19T07:37:28.722 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 202, 2024-09-19T07:37:28.722 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-09-19T07:37:28.723 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-09-19T07:37:28.723 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 2, 2024-09-19T07:37:28.723 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-09-19T07:37:28.723 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-09-19T07:37:28.723 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 2, 2024-09-19T07:37:28.723 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 2, 2024-09-19T07:37:28.723 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-09-19T07:37:28.723 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-09-19T07:37:28.723 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 4194304, 2024-09-19T07:37:28.723 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 4870669, 2024-09-19T07:37:28.723 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 0, 2024-09-19T07:37:28.723 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-09-19T07:37:28.723 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 183, 2024-09-19T07:37:28.723 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.338635086, 2024-09-19T07:37:28.723 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001850464 2024-09-19T07:37:28.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.724 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 0 2024-09-19T07:37:28.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.724 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-09-19T07:37:28.724 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 23, 2024-09-19T07:37:28.724 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 28, 2024-09-19T07:37:28.724 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 5, 2024-09-19T07:37:28.724 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 18, 2024-09-19T07:37:28.724 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 20, 2024-09-19T07:37:28.724 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 2, 2024-09-19T07:37:28.724 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 24, 2024-09-19T07:37:28.724 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 33, 2024-09-19T07:37:28.724 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 9, 2024-09-19T07:37:28.725 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 2, 2024-09-19T07:37:28.725 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 19, 2024-09-19T07:37:28.725 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 17, 2024-09-19T07:37:28.725 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 55400, 2024-09-19T07:37:28.725 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 231712 2024-09-19T07:37:28.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.725 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-09-19T07:37:28.725 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 79, 2024-09-19T07:37:28.725 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-09-19T07:37:28.725 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 64, 2024-09-19T07:37:28.725 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 16, 2024-09-19T07:37:28.725 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-09-19T07:37:28.725 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-09-19T07:37:28.726 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 7, 2024-09-19T07:37:28.726 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.006464313, 2024-09-19T07:37:28.726 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000923473 2024-09-19T07:37:28.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.726 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-09-19T07:37:28.726 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 28, 2024-09-19T07:37:28.726 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.006640079, 2024-09-19T07:37:28.726 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000237145 2024-09-19T07:37:28.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.726 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-09-19T07:37:28.726 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.726 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.726 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.727 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-09-19T07:37:28.727 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.727 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.727 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.727 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-09-19T07:37:28.727 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.727 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.727 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.727 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-09-19T07:37:28.727 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.728 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.728 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.728 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-09-19T07:37:28.728 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1, 2024-09-19T07:37:28.728 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000316881, 2024-09-19T07:37:28.728 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000316881 2024-09-19T07:37:28.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.728 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-09-19T07:37:28.728 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.728 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.728 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.728 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-09-19T07:37:28.729 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.729 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.729 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.729 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-09-19T07:37:28.729 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.729 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.729 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.729 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-09-19T07:37:28.729 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.729 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.730 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.730 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-09-19T07:37:28.730 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 8, 2024-09-19T07:37:28.730 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.011740094, 2024-09-19T07:37:28.731 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001467511 2024-09-19T07:37:28.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.731 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-09-19T07:37:28.731 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.731 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.731 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.731 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-09-19T07:37:28.732 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.732 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.732 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.732 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-09-19T07:37:28.732 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1, 2024-09-19T07:37:28.732 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000208875, 2024-09-19T07:37:28.732 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000208875 2024-09-19T07:37:28.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.732 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-09-19T07:37:28.732 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2, 2024-09-19T07:37:28.732 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.001098706, 2024-09-19T07:37:28.732 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000549353 2024-09-19T07:37:28.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.733 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-09-19T07:37:28.733 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2, 2024-09-19T07:37:28.733 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.002666423, 2024-09-19T07:37:28.733 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001333211 2024-09-19T07:37:28.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.733 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-09-19T07:37:28.733 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.733 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.733 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.733 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-09-19T07:37:28.733 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.734 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.734 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.734 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-09-19T07:37:28.734 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2, 2024-09-19T07:37:28.734 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.007421495, 2024-09-19T07:37:28.734 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.003710747 2024-09-19T07:37:28.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.734 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-09-19T07:37:28.734 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.734 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.734 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.734 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-09-19T07:37:28.734 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.735 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.735 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.735 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-09-19T07:37:28.735 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2, 2024-09-19T07:37:28.735 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.001269623, 2024-09-19T07:37:28.735 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000634811 2024-09-19T07:37:28.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.735 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-09-19T07:37:28.735 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.735 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.735 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.736 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-09-19T07:37:28.736 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.736 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.736 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.736 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-09-19T07:37:28.736 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.736 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.736 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.736 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-09-19T07:37:28.737 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 8, 2024-09-19T07:37:28.737 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.019631824, 2024-09-19T07:37:28.737 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.002453978 2024-09-19T07:37:28.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.737 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-09-19T07:37:28.737 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.737 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.737 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.737 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-09-19T07:37:28.737 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 3, 2024-09-19T07:37:28.737 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.005940541, 2024-09-19T07:37:28.738 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001980180 2024-09-19T07:37:28.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.738 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:37:28.738 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:37:28.738 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-09-19T07:37:28.738 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.738 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.738 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.738 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:28.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.739 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-09-19T07:37:28.739 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 1, 2024-09-19T07:37:28.739 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 3, 2024-09-19T07:37:28.739 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 2, 2024-09-19T07:37:28.739 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 1, 2024-09-19T07:37:28.740 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-09-19T07:37:28.740 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 8, 2024-09-19T07:37:28.740 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 8, 2024-09-19T07:37:28.740 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 215, 2024-09-19T07:37:28.740 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-09-19T07:37:28.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.741 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-09-19T07:37:28.741 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:37:28.741 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-09-19T07:37:28.741 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:37:28.741 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:37:28.741 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:37:28.741 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:37:28.741 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:37:28.741 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:37:28.741 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:37:28.741 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:37:28.741 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:37:28.742 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:37:28.742 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:37:28.742 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:37:28.742 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:37:28.742 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-09-19T07:37:28.742 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:37:28.742 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-09-19T07:37:28.742 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:37:28.742 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:37:28.742 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:37:28.742 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:37:28.743 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:37:28.743 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:37:28.743 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:37:28.743 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-09-19T07:37:28.743 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:37:28.743 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:37:28.743 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:37:28.743 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-09-19T07:37:28.743 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-09-19T07:37:28.743 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-09-19T07:37:28.743 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:37:28.743 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:37:28.743 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:37:28.744 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:37:28.744 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 319848, 2024-09-19T07:37:28.744 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 131, 2024-09-19T07:37:28.744 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 440, 2024-09-19T07:37:28.744 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 5, 2024-09-19T07:37:28.744 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-09-19T07:37:28.744 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-09-19T07:37:28.744 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:37:28.744 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-09-19T07:37:28.744 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:37:28.744 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-09-19T07:37:28.745 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 8952, 2024-09-19T07:37:28.745 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 168, 2024-09-19T07:37:28.745 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:37:28.745 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:37:28.745 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-09-19T07:37:28.745 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-09-19T07:37:28.745 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 98290, 2024-09-19T07:37:28.745 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 714, 2024-09-19T07:37:28.745 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-09-19T07:37:28.745 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-09-19T07:37:28.745 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-09-19T07:37:28.745 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-09-19T07:37:28.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.746 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-09-19T07:37:28.746 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-09-19T07:37:28.746 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-09-19T07:37:28.746 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 5397, 2024-09-19T07:37:28.746 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 694395, 2024-09-19T07:37:28.746 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 37, 2024-09-19T07:37:28.746 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 5397, 2024-09-19T07:37:28.746 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-09-19T07:37:28.746 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 5397, 2024-09-19T07:37:28.746 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 45.496075725, 2024-09-19T07:37:28.746 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.008429882 2024-09-19T07:37:28.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.746 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-09-19T07:37:28.746 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-09-19T07:37:28.747 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 5397, 2024-09-19T07:37:28.747 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 5450 2024-09-19T07:37:28.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.747 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 5397, 2024-09-19T07:37:28.747 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 5, 2024-09-19T07:37:28.747 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 5392, 2024-09-19T07:37:28.747 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-09-19T07:37:28.747 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-09-19T07:37:28.747 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 0, 2024-09-19T07:37:28.747 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 11, 2024-09-19T07:37:28.747 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 5, 2024-09-19T07:37:28.748 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 188, 2024-09-19T07:37:28.748 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 27, 2024-09-19T07:37:28.748 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-09-19T07:37:28.748 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-09-19T07:37:28.748 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-09-19T07:37:28.748 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-09-19T07:37:28.748 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 5127, 2024-09-19T07:37:28.748 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-09-19T07:37:28.748 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-09-19T07:37:28.748 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-09-19T07:37:28.748 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 0, 2024-09-19T07:37:28.748 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 22, 2024-09-19T07:37:28.749 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:37:28.749 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-09-19T07:37:28.749 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:37:28.749 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-09-19T07:37:28.749 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-09-19T07:37:28.749 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-09-19T07:37:28.749 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:37:28.749 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-09-19T07:37:28.749 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:37:28.749 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-09-19T07:37:28.749 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-09-19T07:37:28.749 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-09-19T07:37:28.749 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-09-19T07:37:28.749 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-09-19T07:37:28.750 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-09-19T07:37:28.750 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-09-19T07:37:28.750 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-09-19T07:37:28.750 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-09-19T07:37:28.750 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-09-19T07:37:28.750 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-09-19T07:37:28.750 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-09-19T07:37:28.750 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-09-19T07:37:28.750 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-09-19T07:37:28.750 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-09-19T07:37:28.750 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-09-19T07:37:28.751 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-09-19T07:37:28.751 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 100, 2024-09-19T07:37:28.751 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-09-19T07:37:28.751 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 11, 2024-09-19T07:37:28.751 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 12, 2024-09-19T07:37:28.751 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 12, 2024-09-19T07:37:28.751 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-09-19T07:37:28.751 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-09-19T07:37:28.751 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 57, 2024-09-19T07:37:28.751 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 0, 2024-09-19T07:37:28.751 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 13 2024-09-19T07:37:28.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.751 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-09-19T07:37:28.752 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 1, 2024-09-19T07:37:28.752 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:37:28.752 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-09-19T07:37:28.752 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-09-19T07:37:28.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.752 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-09-19T07:37:28.752 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-09-19T07:37:28.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.752 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-09-19T07:37:28.752 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-09-19T07:37:28.752 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 3585, 2024-09-19T07:37:28.752 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-09-19T07:37:28.752 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 1, 2024-09-19T07:37:28.753 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 4615, 2024-09-19T07:37:28.753 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 5, 2024-09-19T07:37:28.753 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-09-19T07:37:28.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.753 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:37:28.753 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-19T07:37:28.753 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-09-19T07:37:28.753 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-19T07:37:28.753 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 6567, 2024-09-19T07:37:28.753 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 1631127, 2024-09-19T07:37:28.753 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:28.753 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 6567, 2024-09-19T07:37:28.753 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-19T07:37:28.753 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-19T07:37:28.754 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 6567, 2024-09-19T07:37:28.754 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 1631127, 2024-09-19T07:37:28.754 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-19T07:37:28.754 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.754 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.754 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.754 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:28.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.754 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-09-19T07:37:28.754 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-19T07:37:28.754 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-09-19T07:37:28.754 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-19T07:37:28.754 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-19T07:37:28.755 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-19T07:37:28.755 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:28.755 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:28.755 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 5397, 2024-09-19T07:37:28.755 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 694924, 2024-09-19T07:37:28.755 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 5397, 2024-09-19T07:37:28.755 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 694924, 2024-09-19T07:37:28.755 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-19T07:37:28.755 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.755 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.755 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.755 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:28.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.756 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-09-19T07:37:28.756 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-19T07:37:28.756 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-09-19T07:37:28.756 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-19T07:37:28.756 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-19T07:37:28.756 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-19T07:37:28.756 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:28.756 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:28.756 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 5397, 2024-09-19T07:37:28.756 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 5397, 2024-09-19T07:37:28.756 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 5397, 2024-09-19T07:37:28.756 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 5397, 2024-09-19T07:37:28.756 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-19T07:37:28.757 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.757 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.757 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.757 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:28.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.757 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:37:28.757 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-19T07:37:28.757 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-09-19T07:37:28.757 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-19T07:37:28.757 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 5, 2024-09-19T07:37:28.757 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 529, 2024-09-19T07:37:28.757 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:28.758 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 5, 2024-09-19T07:37:28.758 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-19T07:37:28.758 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-19T07:37:28.758 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 5, 2024-09-19T07:37:28.758 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 529, 2024-09-19T07:37:28.758 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-19T07:37:28.758 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.758 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.758 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.758 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:28.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:28.758 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x55f2aff9c960": { 2024-09-19T07:37:28.758 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-19T07:37:28.759 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-09-19T07:37:28.759 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-19T07:37:28.759 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 202, 2024-09-19T07:37:28.759 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 676365, 2024-09-19T07:37:28.759 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:28.759 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 202, 2024-09-19T07:37:28.759 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-19T07:37:28.759 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-19T07:37:28.759 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 183, 2024-09-19T07:37:28.759 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 676365, 2024-09-19T07:37:28.759 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-19T07:37:28.759 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:28.759 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:28.759 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:28.760 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:28.760 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:28.760 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-19T07:37:29.711 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-19T07:37:29.711 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-19T07:37:29.711 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=5 2024-09-19T07:37:29.712 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-19T07:37:29.712 DEBUG:tasks.check_counter:Getting stats from e 2024-09-19T07:37:29.712 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-19T07:37:29.963 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:37:30.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:29 smithi203 ceph-mon[35335]: pgmap v509: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 108 op/s 2024-09-19T07:37:30.291 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:29 smithi189 ceph-mon[34851]: pgmap v509: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 108 op/s 2024-09-19T07:37:30.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:29 smithi159 ceph-mon[30472]: pgmap v509: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 108 op/s 2024-09-19T07:37:30.720 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-19T07:37:30.720 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:37:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 129, 2024-09-19T07:37:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 124, 2024-09-19T07:37:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 475730, 2024-09-19T07:37:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 71956, 2024-09-19T07:37:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 5, 2024-09-19T07:37:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 1, 2024-09-19T07:37:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.020013495, 2024-09-19T07:37:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.007847264, 2024-09-19T07:37:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.016495420, 2024-09-19T07:37:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000018318, 2024-09-19T07:37:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 123, 2024-09-19T07:37:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.007149169, 2024-09-19T07:37:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000058123 2024-09-19T07:37:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 474896, 2024-09-19T07:37:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 71632 2024-09-19T07:37:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:37:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 243, 2024-09-19T07:37:30.723 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 330, 2024-09-19T07:37:30.723 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 59714, 2024-09-19T07:37:30.723 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 290145, 2024-09-19T07:37:30.723 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 5, 2024-09-19T07:37:30.723 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-09-19T07:37:30.723 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.026644569, 2024-09-19T07:37:30.723 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.009764322, 2024-09-19T07:37:30.723 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.005929023, 2024-09-19T07:37:30.723 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.007945244, 2024-09-19T07:37:30.723 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:30.723 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 328, 2024-09-19T07:37:30.723 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.015369414, 2024-09-19T07:37:30.724 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000046857 2024-09-19T07:37:30.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.724 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:30.724 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.724 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.724 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.724 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T07:37:30.724 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 213184 2024-09-19T07:37:30.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.724 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:37:30.724 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 687, 2024-09-19T07:37:30.725 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 687, 2024-09-19T07:37:30.725 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 208197, 2024-09-19T07:37:30.725 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 217700, 2024-09-19T07:37:30.725 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 4, 2024-09-19T07:37:30.725 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 3, 2024-09-19T07:37:30.725 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.070218232, 2024-09-19T07:37:30.725 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.024127139, 2024-09-19T07:37:30.725 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.019619039, 2024-09-19T07:37:30.725 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.020380618, 2024-09-19T07:37:30.725 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:30.725 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 686, 2024-09-19T07:37:30.725 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.033468717, 2024-09-19T07:37:30.725 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000048788 2024-09-19T07:37:30.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.726 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:30.726 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.726 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.726 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.726 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T07:37:30.726 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T07:37:30.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.727 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-09-19T07:37:30.727 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-09-19T07:37:30.727 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-09-19T07:37:30.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.727 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-09-19T07:37:30.727 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-09-19T07:37:30.727 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-09-19T07:37:30.727 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1598, 2024-09-19T07:37:30.727 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.043348687, 2024-09-19T07:37:30.727 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000027126 2024-09-19T07:37:30.727 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:30.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.727 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-09-19T07:37:30.728 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-09-19T07:37:30.728 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-09-19T07:37:30.728 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.728 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.728 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.728 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:30.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.728 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-09-19T07:37:30.728 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 0, 2024-09-19T07:37:30.729 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 0, 2024-09-19T07:37:30.729 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-09-19T07:37:30.729 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.729 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.729 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.729 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-09-19T07:37:30.729 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 0, 2024-09-19T07:37:30.729 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-09-19T07:37:30.729 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-09-19T07:37:30.729 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 0, 2024-09-19T07:37:30.729 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-09-19T07:37:30.730 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-09-19T07:37:30.730 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 0, 2024-09-19T07:37:30.730 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 0, 2024-09-19T07:37:30.730 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 0, 2024-09-19T07:37:30.730 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-09-19T07:37:30.730 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 0, 2024-09-19T07:37:30.730 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 0, 2024-09-19T07:37:30.730 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 0, 2024-09-19T07:37:30.730 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 0, 2024-09-19T07:37:30.730 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 1, 2024-09-19T07:37:30.730 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 0, 2024-09-19T07:37:30.730 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 0, 2024-09-19T07:37:30.730 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-09-19T07:37:30.730 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 0, 2024-09-19T07:37:30.731 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:37:30.731 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-09-19T07:37:30.731 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-09-19T07:37:30.731 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-09-19T07:37:30.731 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-09-19T07:37:30.731 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-09-19T07:37:30.731 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-09-19T07:37:30.731 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-09-19T07:37:30.731 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-09-19T07:37:30.731 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-09-19T07:37:30.731 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:37:30.731 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-09-19T07:37:30.731 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 0, 2024-09-19T07:37:30.732 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 0, 2024-09-19T07:37:30.732 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 0, 2024-09-19T07:37:30.732 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:37:30.732 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-09-19T07:37:30.732 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:37:30.732 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:37:30.732 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:37:30.732 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:37:30.732 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:37:30.732 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-09-19T07:37:30.732 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:37:30.732 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T07:37:30.732 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:37:30.733 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:37:30.733 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:37:30.733 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:37:30.733 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-09-19T07:37:30.733 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:37:30.733 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-09-19T07:37:30.733 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 0 2024-09-19T07:37:30.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.733 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-09-19T07:37:30.733 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-09-19T07:37:30.733 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-09-19T07:37:30.733 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-09-19T07:37:30.733 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 0, 2024-09-19T07:37:30.734 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-09-19T07:37:30.734 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-09-19T07:37:30.734 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-09-19T07:37:30.734 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:37:30.734 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-09-19T07:37:30.734 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-09-19T07:37:30.734 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-09-19T07:37:30.734 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-09-19T07:37:30.734 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-09-19T07:37:30.734 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:37:30.734 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:37:30.734 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-09-19T07:37:30.734 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-09-19T07:37:30.735 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 0, 2024-09-19T07:37:30.735 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 0, 2024-09-19T07:37:30.735 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:37:30.735 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-09-19T07:37:30.735 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-09-19T07:37:30.735 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:37:30.735 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-09-19T07:37:30.735 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-09-19T07:37:30.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.735 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-09-19T07:37:30.735 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-09-19T07:37:30.735 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 0, 2024-09-19T07:37:30.735 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-09-19T07:37:30.735 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-09-19T07:37:30.735 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 4, 2024-09-19T07:37:30.736 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-09-19T07:37:30.736 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-09-19T07:37:30.736 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 0, 2024-09-19T07:37:30.736 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-09-19T07:37:30.736 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-09-19T07:37:30.736 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 2, 2024-09-19T07:37:30.736 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 2, 2024-09-19T07:37:30.736 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-09-19T07:37:30.736 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-09-19T07:37:30.736 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 4194304, 2024-09-19T07:37:30.736 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 4195426, 2024-09-19T07:37:30.736 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 4195285, 2024-09-19T07:37:30.736 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-09-19T07:37:30.736 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.737 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.737 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.737 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 4 2024-09-19T07:37:30.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.737 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-09-19T07:37:30.737 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 3, 2024-09-19T07:37:30.737 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 3, 2024-09-19T07:37:30.737 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 0, 2024-09-19T07:37:30.737 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 1, 2024-09-19T07:37:30.737 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 1, 2024-09-19T07:37:30.737 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 0, 2024-09-19T07:37:30.737 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 0, 2024-09-19T07:37:30.737 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 0, 2024-09-19T07:37:30.738 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 0, 2024-09-19T07:37:30.738 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 0, 2024-09-19T07:37:30.738 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 0, 2024-09-19T07:37:30.738 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 0, 2024-09-19T07:37:30.738 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 48344, 2024-09-19T07:37:30.738 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 215328 2024-09-19T07:37:30.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.738 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-09-19T07:37:30.738 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 0, 2024-09-19T07:37:30.738 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-09-19T07:37:30.738 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 0, 2024-09-19T07:37:30.738 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 0, 2024-09-19T07:37:30.738 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-09-19T07:37:30.739 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-09-19T07:37:30.739 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.739 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.739 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.739 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-09-19T07:37:30.739 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.739 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.739 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.739 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-09-19T07:37:30.739 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.739 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.739 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.740 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-09-19T07:37:30.740 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.740 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.740 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.740 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-09-19T07:37:30.740 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.740 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.740 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.740 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-09-19T07:37:30.740 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.740 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.740 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.741 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-09-19T07:37:30.741 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.741 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.741 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.741 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-09-19T07:37:30.741 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.741 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.741 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.741 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-09-19T07:37:30.741 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.741 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.742 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.742 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-09-19T07:37:30.742 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.742 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.742 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.742 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-09-19T07:37:30.742 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.742 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.742 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.742 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-09-19T07:37:30.742 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.743 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.743 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.743 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-09-19T07:37:30.743 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.743 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.743 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.743 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-09-19T07:37:30.743 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.743 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.743 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.744 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-09-19T07:37:30.744 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.744 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.744 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.744 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-09-19T07:37:30.744 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.744 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.744 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.744 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-09-19T07:37:30.744 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.744 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.745 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.745 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-09-19T07:37:30.745 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.745 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.745 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.745 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-09-19T07:37:30.745 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.745 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.745 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.745 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-09-19T07:37:30.746 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.746 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.746 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.746 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-09-19T07:37:30.746 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.746 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.746 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.746 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-09-19T07:37:30.746 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.746 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.746 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.747 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-09-19T07:37:30.747 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.747 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.747 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.747 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-09-19T07:37:30.747 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.747 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.747 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.747 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-09-19T07:37:30.747 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.748 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.748 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.748 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-09-19T07:37:30.748 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.748 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.748 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.748 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-09-19T07:37:30.748 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.748 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.748 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.748 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-09-19T07:37:30.749 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.749 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.749 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.749 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-09-19T07:37:30.749 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.749 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.749 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.749 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:37:30.749 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:37:30.749 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-09-19T07:37:30.749 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.750 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.750 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.750 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:30.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.750 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-09-19T07:37:30.750 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 0, 2024-09-19T07:37:30.750 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 1, 2024-09-19T07:37:30.750 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 1, 2024-09-19T07:37:30.750 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 1, 2024-09-19T07:37:30.750 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-09-19T07:37:30.750 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 0, 2024-09-19T07:37:30.750 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 0, 2024-09-19T07:37:30.750 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 1552, 2024-09-19T07:37:30.750 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-09-19T07:37:30.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.751 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-09-19T07:37:30.751 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:37:30.751 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-09-19T07:37:30.751 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:37:30.751 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:37:30.751 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:37:30.751 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:37:30.751 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:37:30.751 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:37:30.751 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:37:30.751 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:37:30.751 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:37:30.752 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:37:30.752 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:37:30.752 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:37:30.752 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:37:30.752 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-09-19T07:37:30.752 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:37:30.752 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-09-19T07:37:30.752 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:37:30.752 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:37:30.752 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:37:30.752 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:37:30.752 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:37:30.752 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:37:30.752 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:37:30.753 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-09-19T07:37:30.753 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:37:30.753 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:37:30.753 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:37:30.753 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-09-19T07:37:30.753 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-09-19T07:37:30.753 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-09-19T07:37:30.753 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:37:30.753 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:37:30.753 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:37:30.753 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:37:30.753 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 190041, 2024-09-19T07:37:30.753 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 51, 2024-09-19T07:37:30.753 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:37:30.754 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-09-19T07:37:30.754 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-09-19T07:37:30.754 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-09-19T07:37:30.754 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:37:30.754 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-09-19T07:37:30.754 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:37:30.754 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-09-19T07:37:30.754 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 8952, 2024-09-19T07:37:30.754 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 168, 2024-09-19T07:37:30.754 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:37:30.754 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:37:30.754 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-09-19T07:37:30.754 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-09-19T07:37:30.754 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 8512, 2024-09-19T07:37:30.755 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 13, 2024-09-19T07:37:30.755 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-09-19T07:37:30.755 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-09-19T07:37:30.755 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-09-19T07:37:30.755 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-09-19T07:37:30.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.755 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-09-19T07:37:30.755 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-09-19T07:37:30.755 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-09-19T07:37:30.755 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 926, 2024-09-19T07:37:30.755 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 21, 2024-09-19T07:37:30.755 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 3, 2024-09-19T07:37:30.755 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 926, 2024-09-19T07:37:30.755 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-09-19T07:37:30.755 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 926, 2024-09-19T07:37:30.756 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.571988398, 2024-09-19T07:37:30.756 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000617698 2024-09-19T07:37:30.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.756 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-09-19T07:37:30.756 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-09-19T07:37:30.756 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 926, 2024-09-19T07:37:30.756 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 927 2024-09-19T07:37:30.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.756 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 926, 2024-09-19T07:37:30.756 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 926, 2024-09-19T07:37:30.756 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 0, 2024-09-19T07:37:30.756 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-09-19T07:37:30.756 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-09-19T07:37:30.756 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 579, 2024-09-19T07:37:30.757 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 0, 2024-09-19T07:37:30.757 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 345, 2024-09-19T07:37:30.757 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 0, 2024-09-19T07:37:30.757 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 0, 2024-09-19T07:37:30.757 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-09-19T07:37:30.757 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-09-19T07:37:30.757 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-09-19T07:37:30.757 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-09-19T07:37:30.757 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 0, 2024-09-19T07:37:30.757 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-09-19T07:37:30.757 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-09-19T07:37:30.757 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-09-19T07:37:30.757 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 1, 2024-09-19T07:37:30.757 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 0, 2024-09-19T07:37:30.757 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:37:30.758 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-09-19T07:37:30.758 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:37:30.758 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-09-19T07:37:30.758 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-09-19T07:37:30.758 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-09-19T07:37:30.758 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:37:30.758 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-09-19T07:37:30.758 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:37:30.758 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-09-19T07:37:30.758 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-09-19T07:37:30.758 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-09-19T07:37:30.758 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-09-19T07:37:30.758 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-09-19T07:37:30.758 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-09-19T07:37:30.758 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-09-19T07:37:30.759 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-09-19T07:37:30.759 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-09-19T07:37:30.759 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-09-19T07:37:30.759 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-09-19T07:37:30.759 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-09-19T07:37:30.759 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-09-19T07:37:30.759 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-09-19T07:37:30.759 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-09-19T07:37:30.759 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-09-19T07:37:30.759 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-09-19T07:37:30.759 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 100, 2024-09-19T07:37:30.759 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-09-19T07:37:30.759 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 2, 2024-09-19T07:37:30.759 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 5, 2024-09-19T07:37:30.760 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 5, 2024-09-19T07:37:30.760 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-09-19T07:37:30.760 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-09-19T07:37:30.760 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 0, 2024-09-19T07:37:30.760 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 2, 2024-09-19T07:37:30.760 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 0 2024-09-19T07:37:30.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.760 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-09-19T07:37:30.760 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 0, 2024-09-19T07:37:30.760 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:37:30.760 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-09-19T07:37:30.760 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-09-19T07:37:30.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.760 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-09-19T07:37:30.761 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-09-19T07:37:30.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.761 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-09-19T07:37:30.761 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-09-19T07:37:30.761 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:37:30.761 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-09-19T07:37:30.761 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-09-19T07:37:30.761 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 0, 2024-09-19T07:37:30.761 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-09-19T07:37:30.761 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-09-19T07:37:30.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.761 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:37:30.761 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-19T07:37:30.761 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-09-19T07:37:30.762 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-19T07:37:30.762 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 1055, 2024-09-19T07:37:30.762 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 639110, 2024-09-19T07:37:30.762 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:30.762 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 1055, 2024-09-19T07:37:30.762 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-19T07:37:30.762 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-19T07:37:30.762 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 1055, 2024-09-19T07:37:30.762 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 639110, 2024-09-19T07:37:30.762 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-19T07:37:30.762 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.762 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.762 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.762 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:30.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.763 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-09-19T07:37:30.763 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-19T07:37:30.763 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-09-19T07:37:30.763 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-19T07:37:30.763 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-19T07:37:30.763 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-19T07:37:30.763 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:30.763 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:30.763 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 926, 2024-09-19T07:37:30.763 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 1127, 2024-09-19T07:37:30.763 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 926, 2024-09-19T07:37:30.763 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 1127, 2024-09-19T07:37:30.763 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-19T07:37:30.763 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.764 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.764 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.764 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:30.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.764 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-09-19T07:37:30.764 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-19T07:37:30.764 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-09-19T07:37:30.764 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-19T07:37:30.764 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-19T07:37:30.764 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-19T07:37:30.764 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:30.764 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:30.764 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 926, 2024-09-19T07:37:30.765 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 926, 2024-09-19T07:37:30.765 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 926, 2024-09-19T07:37:30.765 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 926, 2024-09-19T07:37:30.765 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-19T07:37:30.765 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.765 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.765 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.765 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:30.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.765 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:37:30.765 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-19T07:37:30.765 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-09-19T07:37:30.765 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-19T07:37:30.765 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-19T07:37:30.765 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-19T07:37:30.766 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:30.766 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:30.766 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-19T07:37:30.766 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-19T07:37:30.766 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-09-19T07:37:30.766 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-09-19T07:37:30.766 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-19T07:37:30.766 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.766 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.766 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.766 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:30.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:30.766 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x560977ca6b40": { 2024-09-19T07:37:30.766 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-19T07:37:30.767 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-09-19T07:37:30.767 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-19T07:37:30.767 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-19T07:37:30.767 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-19T07:37:30.767 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:30.767 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:30.767 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-19T07:37:30.767 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-19T07:37:30.767 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-09-19T07:37:30.767 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-09-19T07:37:30.767 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-19T07:37:30.767 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:30.767 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:30.768 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:30.768 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:30.768 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:30.768 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-19T07:37:31.588 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-19T07:37:31.589 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-19T07:37:31.589 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-19T07:37:31.589 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-19T07:37:31.589 DEBUG:tasks.check_counter:Getting stats from h 2024-09-19T07:37:31.589 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-19T07:37:31.835 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:37:32.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:31 smithi203 ceph-mon[35335]: pgmap v510: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 47 op/s 2024-09-19T07:37:32.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:31 smithi189 ceph-mon[34851]: pgmap v510: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 47 op/s 2024-09-19T07:37:32.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:31 smithi159 ceph-mon[30472]: pgmap v510: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 47 op/s 2024-09-19T07:37:32.919 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-19T07:37:32.919 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:37:32.919 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 171, 2024-09-19T07:37:32.919 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 238, 2024-09-19T07:37:32.919 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 47386, 2024-09-19T07:37:32.919 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 65859, 2024-09-19T07:37:32.919 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 8, 2024-09-19T07:37:32.920 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 2, 2024-09-19T07:37:32.920 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.027826665, 2024-09-19T07:37:32.920 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.011698467, 2024-09-19T07:37:32.920 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.012493365, 2024-09-19T07:37:32.920 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.002042745, 2024-09-19T07:37:32.920 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:32.920 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 238, 2024-09-19T07:37:32.920 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.029071106, 2024-09-19T07:37:32.920 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000122147 2024-09-19T07:37:32.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.920 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:32.920 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 366, 2024-09-19T07:37:32.921 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000078087, 2024-09-19T07:37:32.921 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000213 2024-09-19T07:37:32.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.921 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:37:32.921 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:37:32.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.921 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:37:32.921 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 40, 2024-09-19T07:37:32.921 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 122, 2024-09-19T07:37:32.921 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 16751, 2024-09-19T07:37:32.921 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 261367, 2024-09-19T07:37:32.922 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 9, 2024-09-19T07:37:32.922 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 5, 2024-09-19T07:37:32.922 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.011898392, 2024-09-19T07:37:32.922 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.004845683, 2024-09-19T07:37:32.922 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.005952508, 2024-09-19T07:37:32.922 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000299493, 2024-09-19T07:37:32.922 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:32.922 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 123, 2024-09-19T07:37:32.922 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.022036599, 2024-09-19T07:37:32.922 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000179159 2024-09-19T07:37:32.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.923 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:32.923 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 37, 2024-09-19T07:37:32.923 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000151016, 2024-09-19T07:37:32.923 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000004081 2024-09-19T07:37:32.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.923 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-19T07:37:32.923 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 215072 2024-09-19T07:37:32.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.923 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:37:32.923 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 171, 2024-09-19T07:37:32.923 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 168, 2024-09-19T07:37:32.923 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 487572, 2024-09-19T07:37:32.923 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 104265, 2024-09-19T07:37:32.924 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 9, 2024-09-19T07:37:32.924 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 8, 2024-09-19T07:37:32.924 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.021721034, 2024-09-19T07:37:32.924 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.007924654, 2024-09-19T07:37:32.924 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.018632339, 2024-09-19T07:37:32.924 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000128022, 2024-09-19T07:37:32.924 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:32.924 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 163, 2024-09-19T07:37:32.924 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.011704294, 2024-09-19T07:37:32.924 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000071805 2024-09-19T07:37:32.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.924 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:32.924 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 13, 2024-09-19T07:37:32.925 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000001541, 2024-09-19T07:37:32.925 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000118 2024-09-19T07:37:32.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.925 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 485888, 2024-09-19T07:37:32.925 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 77072 2024-09-19T07:37:32.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.925 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-09-19T07:37:32.925 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-09-19T07:37:32.925 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-09-19T07:37:32.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.925 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-09-19T07:37:32.925 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-09-19T07:37:32.925 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-09-19T07:37:32.926 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 51, 2024-09-19T07:37:32.926 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.001156674, 2024-09-19T07:37:32.926 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000022679 2024-09-19T07:37:32.926 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:32.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.926 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-09-19T07:37:32.926 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-09-19T07:37:32.926 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-09-19T07:37:32.926 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2, 2024-09-19T07:37:32.927 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000006924, 2024-09-19T07:37:32.927 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000003462 2024-09-19T07:37:32.927 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:32.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.927 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-09-19T07:37:32.927 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 2, 2024-09-19T07:37:32.927 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 1, 2024-09-19T07:37:32.927 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-09-19T07:37:32.927 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1, 2024-09-19T07:37:32.927 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000264115, 2024-09-19T07:37:32.927 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000264115 2024-09-19T07:37:32.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.927 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-09-19T07:37:32.928 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 1, 2024-09-19T07:37:32.928 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-09-19T07:37:32.928 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-09-19T07:37:32.928 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 11, 2024-09-19T07:37:32.928 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-09-19T07:37:32.928 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-09-19T07:37:32.929 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 14, 2024-09-19T07:37:32.929 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 1, 2024-09-19T07:37:32.929 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 13, 2024-09-19T07:37:32.929 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-09-19T07:37:32.929 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 12, 2024-09-19T07:37:32.929 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 0, 2024-09-19T07:37:32.929 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 0, 2024-09-19T07:37:32.929 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 0, 2024-09-19T07:37:32.929 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 2, 2024-09-19T07:37:32.929 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 8, 2024-09-19T07:37:32.930 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 4, 2024-09-19T07:37:32.930 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-09-19T07:37:32.930 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 3, 2024-09-19T07:37:32.930 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:37:32.930 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-09-19T07:37:32.930 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-09-19T07:37:32.930 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-09-19T07:37:32.930 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-09-19T07:37:32.930 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-09-19T07:37:32.930 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-09-19T07:37:32.930 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-09-19T07:37:32.930 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-09-19T07:37:32.930 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-09-19T07:37:32.931 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:37:32.931 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-09-19T07:37:32.931 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 3, 2024-09-19T07:37:32.931 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 117, 2024-09-19T07:37:32.931 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 0, 2024-09-19T07:37:32.931 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:37:32.931 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-09-19T07:37:32.931 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:37:32.931 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:37:32.931 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:37:32.932 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:37:32.932 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:37:32.932 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-09-19T07:37:32.932 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:37:32.932 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T07:37:32.932 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:37:32.932 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:37:32.932 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:37:32.932 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:37:32.932 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-09-19T07:37:32.932 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:37:32.932 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-09-19T07:37:32.932 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 1 2024-09-19T07:37:32.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.933 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-09-19T07:37:32.933 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-09-19T07:37:32.933 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 6, 2024-09-19T07:37:32.933 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 4, 2024-09-19T07:37:32.933 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 6, 2024-09-19T07:37:32.933 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-09-19T07:37:32.933 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-09-19T07:37:32.933 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-09-19T07:37:32.933 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:37:32.934 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-09-19T07:37:32.934 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-09-19T07:37:32.934 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-09-19T07:37:32.934 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-09-19T07:37:32.934 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-09-19T07:37:32.934 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:37:32.934 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:37:32.934 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-09-19T07:37:32.934 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-09-19T07:37:32.934 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 0, 2024-09-19T07:37:32.934 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 0, 2024-09-19T07:37:32.934 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:37:32.934 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-09-19T07:37:32.934 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-09-19T07:37:32.935 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:37:32.935 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-09-19T07:37:32.935 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-09-19T07:37:32.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.935 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-09-19T07:37:32.935 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-09-19T07:37:32.935 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 4, 2024-09-19T07:37:32.935 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-09-19T07:37:32.935 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-09-19T07:37:32.935 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 4, 2024-09-19T07:37:32.935 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-09-19T07:37:32.935 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-09-19T07:37:32.935 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 2, 2024-09-19T07:37:32.936 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-09-19T07:37:32.936 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-09-19T07:37:32.936 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 2, 2024-09-19T07:37:32.936 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 2, 2024-09-19T07:37:32.936 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-09-19T07:37:32.936 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-09-19T07:37:32.936 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 4194304, 2024-09-19T07:37:32.936 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 4195426, 2024-09-19T07:37:32.936 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 0, 2024-09-19T07:37:32.936 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-09-19T07:37:32.936 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 3, 2024-09-19T07:37:32.936 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.005589393, 2024-09-19T07:37:32.937 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001863131 2024-09-19T07:37:32.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.937 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 0 2024-09-19T07:37:32.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.937 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-09-19T07:37:32.937 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 17, 2024-09-19T07:37:32.937 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 17, 2024-09-19T07:37:32.937 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 0, 2024-09-19T07:37:32.937 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 16, 2024-09-19T07:37:32.937 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 17, 2024-09-19T07:37:32.937 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 1, 2024-09-19T07:37:32.937 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 14, 2024-09-19T07:37:32.937 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 14, 2024-09-19T07:37:32.937 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 0, 2024-09-19T07:37:32.938 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 0, 2024-09-19T07:37:32.938 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 1, 2024-09-19T07:37:32.938 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 1, 2024-09-19T07:37:32.938 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 51788, 2024-09-19T07:37:32.938 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 223520 2024-09-19T07:37:32.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.939 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-09-19T07:37:32.939 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 3, 2024-09-19T07:37:32.939 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-09-19T07:37:32.939 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 2, 2024-09-19T07:37:32.939 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 4, 2024-09-19T07:37:32.939 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-09-19T07:37:32.939 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-09-19T07:37:32.939 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.939 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.939 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.940 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-09-19T07:37:32.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1, 2024-09-19T07:37:32.940 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000264115, 2024-09-19T07:37:32.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000264115 2024-09-19T07:37:32.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.940 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-09-19T07:37:32.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.940 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.940 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-09-19T07:37:32.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.941 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.941 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-09-19T07:37:32.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.941 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.941 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-09-19T07:37:32.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.941 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.942 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-09-19T07:37:32.942 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.942 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.942 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.942 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-09-19T07:37:32.942 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.942 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.942 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.942 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-09-19T07:37:32.942 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.942 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.943 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.943 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-09-19T07:37:32.943 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.943 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.943 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.943 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-09-19T07:37:32.943 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.943 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.943 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.943 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-09-19T07:37:32.943 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.944 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.944 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.945 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-09-19T07:37:32.945 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.945 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.945 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.945 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-09-19T07:37:32.945 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.945 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.945 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.945 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-09-19T07:37:32.946 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.946 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.946 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.946 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-09-19T07:37:32.946 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.946 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.946 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.946 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-09-19T07:37:32.946 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.946 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.947 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.947 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-09-19T07:37:32.947 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.947 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.947 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.947 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-09-19T07:37:32.947 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.947 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.947 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.948 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-09-19T07:37:32.948 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.948 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.948 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.948 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-09-19T07:37:32.948 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.948 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.948 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.948 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-09-19T07:37:32.948 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.948 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.949 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.949 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-09-19T07:37:32.949 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.949 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.949 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.949 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-09-19T07:37:32.949 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.949 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.949 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.949 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-09-19T07:37:32.950 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.950 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.950 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.950 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-09-19T07:37:32.950 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.950 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.950 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.950 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-09-19T07:37:32.951 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.951 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.951 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.951 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-09-19T07:37:32.951 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.951 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.951 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.951 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-09-19T07:37:32.951 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.951 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.951 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.952 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:37:32.952 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:37:32.952 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-09-19T07:37:32.952 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.952 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.952 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.952 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:32.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.952 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-09-19T07:37:32.952 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 0, 2024-09-19T07:37:32.952 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 1, 2024-09-19T07:37:32.953 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 1, 2024-09-19T07:37:32.953 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 0, 2024-09-19T07:37:32.953 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-09-19T07:37:32.953 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 1, 2024-09-19T07:37:32.953 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 1, 2024-09-19T07:37:32.953 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 88, 2024-09-19T07:37:32.953 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-09-19T07:37:32.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.953 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-09-19T07:37:32.953 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:37:32.953 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-09-19T07:37:32.953 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:37:32.953 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:37:32.954 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:37:32.954 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:37:32.954 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:37:32.954 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:37:32.954 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:37:32.954 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:37:32.954 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:37:32.954 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:37:32.954 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:37:32.954 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:37:32.954 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:37:32.954 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-09-19T07:37:32.954 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:37:32.955 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-09-19T07:37:32.955 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:37:32.955 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:37:32.955 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:37:32.955 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:37:32.955 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:37:32.955 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:37:32.955 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:37:32.955 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-09-19T07:37:32.955 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:37:32.955 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:37:32.955 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:37:32.955 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-09-19T07:37:32.956 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-09-19T07:37:32.956 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-09-19T07:37:32.956 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:37:32.956 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:37:32.956 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:37:32.956 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:37:32.956 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 230713, 2024-09-19T07:37:32.956 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 75, 2024-09-19T07:37:32.956 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:37:32.956 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-09-19T07:37:32.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-09-19T07:37:32.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-09-19T07:37:32.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:37:32.957 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-09-19T07:37:32.957 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:37:32.957 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-09-19T07:37:32.957 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 8952, 2024-09-19T07:37:32.957 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 168, 2024-09-19T07:37:32.957 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:37:32.957 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:37:32.957 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-09-19T07:37:32.957 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-09-19T07:37:32.957 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 69056, 2024-09-19T07:37:32.957 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 255, 2024-09-19T07:37:32.957 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-09-19T07:37:32.957 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-09-19T07:37:32.958 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-09-19T07:37:32.958 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-09-19T07:37:32.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.958 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-09-19T07:37:32.958 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-09-19T07:37:32.958 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-09-19T07:37:32.958 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 38, 2024-09-19T07:37:32.958 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 11334, 2024-09-19T07:37:32.958 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 33, 2024-09-19T07:37:32.958 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 38, 2024-09-19T07:37:32.958 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-09-19T07:37:32.958 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 38, 2024-09-19T07:37:32.958 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.234995297, 2024-09-19T07:37:32.958 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.006184086 2024-09-19T07:37:32.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.959 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-09-19T07:37:32.959 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-09-19T07:37:32.959 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 38, 2024-09-19T07:37:32.959 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 60 2024-09-19T07:37:32.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.959 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 38, 2024-09-19T07:37:32.959 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 0, 2024-09-19T07:37:32.959 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 38, 2024-09-19T07:37:32.959 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-09-19T07:37:32.959 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-09-19T07:37:32.959 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 0, 2024-09-19T07:37:32.960 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 10, 2024-09-19T07:37:32.960 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 0, 2024-09-19T07:37:32.960 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 3, 2024-09-19T07:37:32.960 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 6, 2024-09-19T07:37:32.960 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-09-19T07:37:32.960 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-09-19T07:37:32.960 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-09-19T07:37:32.960 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-09-19T07:37:32.960 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 6, 2024-09-19T07:37:32.960 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-09-19T07:37:32.960 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-09-19T07:37:32.960 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-09-19T07:37:32.960 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 0, 2024-09-19T07:37:32.961 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 20, 2024-09-19T07:37:32.961 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:37:32.961 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-09-19T07:37:32.961 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:37:32.961 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-09-19T07:37:32.961 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-09-19T07:37:32.961 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-09-19T07:37:32.961 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:37:32.961 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-09-19T07:37:32.961 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:37:32.961 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-09-19T07:37:32.961 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-09-19T07:37:32.961 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-09-19T07:37:32.961 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-09-19T07:37:32.962 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-09-19T07:37:32.962 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-09-19T07:37:32.962 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-09-19T07:37:32.962 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-09-19T07:37:32.962 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-09-19T07:37:32.962 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-09-19T07:37:32.962 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-09-19T07:37:32.962 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-09-19T07:37:32.962 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-09-19T07:37:32.962 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-09-19T07:37:32.962 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-09-19T07:37:32.962 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-09-19T07:37:32.962 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-09-19T07:37:32.962 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 100, 2024-09-19T07:37:32.963 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-09-19T07:37:32.963 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 2, 2024-09-19T07:37:32.963 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 8, 2024-09-19T07:37:32.963 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 8, 2024-09-19T07:37:32.963 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-09-19T07:37:32.963 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-09-19T07:37:32.963 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 14, 2024-09-19T07:37:32.963 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 0, 2024-09-19T07:37:32.963 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 1 2024-09-19T07:37:32.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.964 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-09-19T07:37:32.964 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 1, 2024-09-19T07:37:32.964 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:37:32.964 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-09-19T07:37:32.964 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-09-19T07:37:32.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.964 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-09-19T07:37:32.964 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-09-19T07:37:32.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.964 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-09-19T07:37:32.964 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-09-19T07:37:32.964 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:37:32.964 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-09-19T07:37:32.965 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-09-19T07:37:32.965 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 0, 2024-09-19T07:37:32.965 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-09-19T07:37:32.965 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-09-19T07:37:32.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.965 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:37:32.965 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-19T07:37:32.965 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-09-19T07:37:32.965 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-19T07:37:32.965 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 378, 2024-09-19T07:37:32.965 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 506477, 2024-09-19T07:37:32.965 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:32.965 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 378, 2024-09-19T07:37:32.965 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-19T07:37:32.966 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-19T07:37:32.966 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 378, 2024-09-19T07:37:32.966 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 506477, 2024-09-19T07:37:32.966 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-19T07:37:32.966 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.966 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.966 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.966 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:32.966 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.966 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-09-19T07:37:32.966 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-19T07:37:32.966 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-09-19T07:37:32.966 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-19T07:37:32.967 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-19T07:37:32.967 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-19T07:37:32.967 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:32.967 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:32.967 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 38, 2024-09-19T07:37:32.967 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 11334, 2024-09-19T07:37:32.967 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 38, 2024-09-19T07:37:32.967 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 11334, 2024-09-19T07:37:32.967 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-19T07:37:32.967 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.967 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.967 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.968 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:32.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.968 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-09-19T07:37:32.968 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-19T07:37:32.968 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-09-19T07:37:32.968 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-19T07:37:32.968 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-19T07:37:32.968 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-19T07:37:32.968 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:32.968 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:32.968 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 38, 2024-09-19T07:37:32.968 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 38, 2024-09-19T07:37:32.968 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 38, 2024-09-19T07:37:32.969 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 38, 2024-09-19T07:37:32.969 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-19T07:37:32.969 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.969 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.969 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.969 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:32.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.969 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:37:32.969 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-19T07:37:32.969 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-09-19T07:37:32.969 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-19T07:37:32.969 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-19T07:37:32.969 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-19T07:37:32.969 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:32.970 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:32.970 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-19T07:37:32.970 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-19T07:37:32.970 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-09-19T07:37:32.970 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-09-19T07:37:32.970 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-19T07:37:32.970 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.970 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.970 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.970 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:32.970 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:32.970 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x55e449d7a960": { 2024-09-19T07:37:32.970 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-19T07:37:32.971 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-09-19T07:37:32.971 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-19T07:37:32.971 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 4, 2024-09-19T07:37:32.971 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 1122, 2024-09-19T07:37:32.971 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:32.971 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 4, 2024-09-19T07:37:32.971 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-19T07:37:32.971 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-19T07:37:32.971 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 3, 2024-09-19T07:37:32.971 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 1122, 2024-09-19T07:37:32.971 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-19T07:37:32.971 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:32.971 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:32.972 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:32.972 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:32.972 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:32.972 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-19T07:37:33.793 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-19T07:37:33.794 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-19T07:37:33.794 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-19T07:37:33.794 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=6 2024-09-19T07:37:33.794 DEBUG:tasks.check_counter:Getting stats from k 2024-09-19T07:37:33.794 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-19T07:37:34.041 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:37:34.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:33 smithi189 ceph-mon[34851]: pgmap v511: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-19T07:37:34.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:33 smithi203 ceph-mon[35335]: pgmap v511: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-19T07:37:34.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:33 smithi159 ceph-mon[30472]: pgmap v511: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-19T07:37:34.586 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-19T07:37:34.586 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:37:34.586 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 123, 2024-09-19T07:37:34.586 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 120, 2024-09-19T07:37:34.586 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 452272, 2024-09-19T07:37:34.586 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 73008, 2024-09-19T07:37:34.586 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 4, 2024-09-19T07:37:34.586 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 0, 2024-09-19T07:37:34.586 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.019894594, 2024-09-19T07:37:34.586 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.008043052, 2024-09-19T07:37:34.586 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.015755345, 2024-09-19T07:37:34.587 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:34.587 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:34.587 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 119, 2024-09-19T07:37:34.587 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.007406816, 2024-09-19T07:37:34.587 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000062242 2024-09-19T07:37:34.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:34.587 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:34.587 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:34.587 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:34.587 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:34.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:34.587 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 452272, 2024-09-19T07:37:34.587 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 73008 2024-09-19T07:37:34.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:34.588 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:37:34.588 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1, 2024-09-19T07:37:34.588 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 91, 2024-09-19T07:37:34.588 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 160, 2024-09-19T07:37:34.588 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 94544, 2024-09-19T07:37:34.588 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 3, 2024-09-19T07:37:34.588 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 2, 2024-09-19T07:37:34.588 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.005559453, 2024-09-19T07:37:34.588 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.003771600, 2024-09-19T07:37:34.588 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.000798672, 2024-09-19T07:37:34.588 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:34.588 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:34.588 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 89, 2024-09-19T07:37:34.589 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.005485571, 2024-09-19T07:37:34.589 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000061635 2024-09-19T07:37:34.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:34.589 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:34.589 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:34.589 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:34.589 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:34.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:34.589 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T07:37:34.589 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 94544 2024-09-19T07:37:34.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:34.589 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:37:34.589 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 4, 2024-09-19T07:37:34.590 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 4, 2024-09-19T07:37:34.590 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 3024, 2024-09-19T07:37:34.590 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 448, 2024-09-19T07:37:34.590 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 2, 2024-09-19T07:37:34.590 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 1, 2024-09-19T07:37:34.590 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.002221631, 2024-09-19T07:37:34.590 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.000038276, 2024-09-19T07:37:34.590 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.002180391, 2024-09-19T07:37:34.590 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:34.590 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:34.590 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 3, 2024-09-19T07:37:34.590 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.004008915, 2024-09-19T07:37:34.590 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001336305 2024-09-19T07:37:34.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:34.591 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:34.591 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:34.591 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:34.591 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:34.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:34.591 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T07:37:34.591 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T07:37:34.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:34.591 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-09-19T07:37:34.591 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 0, 2024-09-19T07:37:34.591 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-09-19T07:37:34.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:34.591 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-09-19T07:37:34.591 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:37:34.592 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-09-19T07:37:34.592 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:37:34.592 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:37:34.592 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:37:34.592 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:37:34.592 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:37:34.592 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:37:34.592 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:37:34.592 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:37:34.592 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:37:34.592 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:37:34.592 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:37:34.592 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:37:34.592 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:37:34.593 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-09-19T07:37:34.593 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:37:34.593 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-09-19T07:37:34.593 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:37:34.593 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:37:34.593 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:37:34.593 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:37:34.593 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:37:34.593 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:37:34.593 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:37:34.593 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-09-19T07:37:34.593 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:37:34.593 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:37:34.594 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:37:34.594 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-09-19T07:37:34.594 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-09-19T07:37:34.594 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-09-19T07:37:34.594 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:37:34.594 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:37:34.594 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:37:34.594 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:37:34.594 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 148840, 2024-09-19T07:37:34.594 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 28, 2024-09-19T07:37:34.594 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:37:34.594 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-09-19T07:37:34.594 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-09-19T07:37:34.594 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-09-19T07:37:34.594 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:37:34.595 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-09-19T07:37:34.595 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:37:34.595 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-09-19T07:37:34.595 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 0, 2024-09-19T07:37:34.595 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 0, 2024-09-19T07:37:34.595 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:37:34.595 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:37:34.595 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-09-19T07:37:34.595 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-09-19T07:37:34.595 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 928, 2024-09-19T07:37:34.595 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 2, 2024-09-19T07:37:34.595 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-09-19T07:37:34.596 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-09-19T07:37:34.596 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-09-19T07:37:34.596 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-09-19T07:37:34.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-19T07:37:34.596 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:37:34.596 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-19T07:37:34.596 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-09-19T07:37:34.596 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-19T07:37:34.596 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 124, 2024-09-19T07:37:34.596 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 434915, 2024-09-19T07:37:34.596 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:34.596 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 124, 2024-09-19T07:37:34.596 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-19T07:37:34.597 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-19T07:37:34.597 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 124, 2024-09-19T07:37:34.597 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 434915, 2024-09-19T07:37:34.597 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-19T07:37:34.597 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-19T07:37:34.598 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-19T07:37:34.598 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-19T07:37:34.598 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:34.598 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-19T07:37:34.598 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-19T07:37:35.321 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-19T07:37:35.321 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-19T07:37:35.321 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-19T07:37:35.321 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-19T07:37:35.321 DEBUG:tasks.check_counter:Getting stats from c 2024-09-19T07:37:35.321 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-19T07:37:35.581 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:37:36.111 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-09-19T07:37:36.112 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:37:36.112 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 769, 2024-09-19T07:37:36.112 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 769, 2024-09-19T07:37:36.112 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 904664, 2024-09-19T07:37:36.112 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 243728, 2024-09-19T07:37:36.113 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 6, 2024-09-19T07:37:36.113 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 2, 2024-09-19T07:37:36.113 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.070706867, 2024-09-19T07:37:36.113 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.021354201, 2024-09-19T07:37:36.113 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.021005347, 2024-09-19T07:37:36.113 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.021259277, 2024-09-19T07:37:36.113 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:36.113 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 768, 2024-09-19T07:37:36.113 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.031748176, 2024-09-19T07:37:36.113 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000041338 2024-09-19T07:37:36.114 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.114 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:36.114 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.114 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.114 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.114 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.114 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T07:37:36.114 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T07:37:36.114 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.114 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:37:36.114 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 132, 2024-09-19T07:37:36.114 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 127, 2024-09-19T07:37:36.115 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 476376, 2024-09-19T07:37:36.115 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 72970, 2024-09-19T07:37:36.115 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 4, 2024-09-19T07:37:36.115 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 3, 2024-09-19T07:37:36.115 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.020212037, 2024-09-19T07:37:36.115 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.008458553, 2024-09-19T07:37:36.115 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.016024944, 2024-09-19T07:37:36.115 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000115075, 2024-09-19T07:37:36.115 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:36.115 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 126, 2024-09-19T07:37:36.115 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.011396455, 2024-09-19T07:37:36.116 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000090448 2024-09-19T07:37:36.116 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.116 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:36.116 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.116 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.116 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.116 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.116 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 474720, 2024-09-19T07:37:36.117 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 71632 2024-09-19T07:37:36.117 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.117 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:37:36.117 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 346, 2024-09-19T07:37:36.117 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 433, 2024-09-19T07:37:36.117 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 85101, 2024-09-19T07:37:36.117 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 324511, 2024-09-19T07:37:36.117 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 4, 2024-09-19T07:37:36.117 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 3, 2024-09-19T07:37:36.118 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.037235529, 2024-09-19T07:37:36.118 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.015157082, 2024-09-19T07:37:36.118 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.007770415, 2024-09-19T07:37:36.118 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.010521003, 2024-09-19T07:37:36.118 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:36.118 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 431, 2024-09-19T07:37:36.118 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.017631951, 2024-09-19T07:37:36.118 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000040909 2024-09-19T07:37:36.118 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.118 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:36.119 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.119 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.119 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.119 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.119 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T07:37:36.119 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 214800 2024-09-19T07:37:36.119 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.119 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2024-09-19T07:37:36.119 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2024-09-19T07:37:36.119 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2024-09-19T07:37:36.120 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.120 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2024-09-19T07:37:36.120 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-09-19T07:37:36.120 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-09-19T07:37:36.120 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1846, 2024-09-19T07:37:36.120 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.047737987, 2024-09-19T07:37:36.120 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000025860 2024-09-19T07:37:36.120 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:36.120 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.121 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2024-09-19T07:37:36.121 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-09-19T07:37:36.121 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-09-19T07:37:36.121 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.121 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.121 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.121 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:36.122 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.122 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2024-09-19T07:37:36.122 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 0, 2024-09-19T07:37:36.122 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 0, 2024-09-19T07:37:36.122 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2024-09-19T07:37:36.122 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.122 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.123 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.123 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.123 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2024-09-19T07:37:36.123 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 0, 2024-09-19T07:37:36.123 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 0, 2024-09-19T07:37:36.123 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2024-09-19T07:37:36.123 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 0, 2024-09-19T07:37:36.123 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 0, 2024-09-19T07:37:36.123 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 0, 2024-09-19T07:37:36.124 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 19, 2024-09-19T07:37:36.124 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 0, 2024-09-19T07:37:36.124 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 19, 2024-09-19T07:37:36.124 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 0, 2024-09-19T07:37:36.124 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 19, 2024-09-19T07:37:36.124 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 0, 2024-09-19T07:37:36.124 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 0, 2024-09-19T07:37:36.124 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 0, 2024-09-19T07:37:36.124 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 2, 2024-09-19T07:37:36.124 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 0, 2024-09-19T07:37:36.124 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 0, 2024-09-19T07:37:36.124 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 0, 2024-09-19T07:37:36.124 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 0, 2024-09-19T07:37:36.125 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:37:36.125 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2024-09-19T07:37:36.125 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 0, 2024-09-19T07:37:36.125 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 0, 2024-09-19T07:37:36.125 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2024-09-19T07:37:36.125 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 0, 2024-09-19T07:37:36.125 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 0, 2024-09-19T07:37:36.125 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 0, 2024-09-19T07:37:36.125 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 0, 2024-09-19T07:37:36.125 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2024-09-19T07:37:36.125 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:37:36.125 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 0, 2024-09-19T07:37:36.125 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 4, 2024-09-19T07:37:36.126 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 234, 2024-09-19T07:37:36.126 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 0, 2024-09-19T07:37:36.126 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:37:36.126 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2024-09-19T07:37:36.126 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:37:36.126 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:37:36.126 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:37:36.126 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:37:36.126 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:37:36.126 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 0, 2024-09-19T07:37:36.126 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:37:36.126 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T07:37:36.126 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:37:36.126 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:37:36.127 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:37:36.127 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:37:36.127 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 0, 2024-09-19T07:37:36.127 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:37:36.127 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 0, 2024-09-19T07:37:36.127 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 0 2024-09-19T07:37:36.127 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.127 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2024-09-19T07:37:36.127 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 0, 2024-09-19T07:37:36.127 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 0, 2024-09-19T07:37:36.127 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 0, 2024-09-19T07:37:36.127 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 0, 2024-09-19T07:37:36.127 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 0, 2024-09-19T07:37:36.127 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 0, 2024-09-19T07:37:36.128 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2024-09-19T07:37:36.128 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:37:36.128 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 0, 2024-09-19T07:37:36.128 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 0, 2024-09-19T07:37:36.128 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 0, 2024-09-19T07:37:36.128 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2024-09-19T07:37:36.128 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2024-09-19T07:37:36.128 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:37:36.128 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:37:36.129 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2024-09-19T07:37:36.129 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2024-09-19T07:37:36.129 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_path": 0, 2024-09-19T07:37:36.129 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_inode": 0, 2024-09-19T07:37:36.129 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:37:36.129 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 0, 2024-09-19T07:37:36.129 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2024-09-19T07:37:36.129 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:37:36.129 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2024-09-19T07:37:36.129 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0 2024-09-19T07:37:36.129 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.129 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2024-09-19T07:37:36.129 INFO:teuthology.orchestra.run.smithi203.stdout: "evlrg": 0, 2024-09-19T07:37:36.129 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 0, 2024-09-19T07:37:36.130 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 0, 2024-09-19T07:37:36.130 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 0, 2024-09-19T07:37:36.130 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 202, 2024-09-19T07:37:36.130 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2024-09-19T07:37:36.130 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 0, 2024-09-19T07:37:36.130 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 0, 2024-09-19T07:37:36.130 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 0, 2024-09-19T07:37:36.130 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 0, 2024-09-19T07:37:36.130 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 2, 2024-09-19T07:37:36.130 INFO:teuthology.orchestra.run.smithi203.stdout: "segmjr": 2, 2024-09-19T07:37:36.130 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2024-09-19T07:37:36.130 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 0, 2024-09-19T07:37:36.130 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 4194304, 2024-09-19T07:37:36.131 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 4870669, 2024-09-19T07:37:36.131 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 4870070, 2024-09-19T07:37:36.131 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2024-09-19T07:37:36.131 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.131 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.131 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.131 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.131 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 202 2024-09-19T07:37:36.131 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.131 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2024-09-19T07:37:36.132 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 14, 2024-09-19T07:37:36.132 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 19, 2024-09-19T07:37:36.132 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 5, 2024-09-19T07:37:36.132 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 9, 2024-09-19T07:37:36.132 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 11, 2024-09-19T07:37:36.132 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 2, 2024-09-19T07:37:36.132 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 19, 2024-09-19T07:37:36.132 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 23, 2024-09-19T07:37:36.132 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 4, 2024-09-19T07:37:36.132 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 0, 2024-09-19T07:37:36.132 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 0, 2024-09-19T07:37:36.132 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 0, 2024-09-19T07:37:36.132 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 48104, 2024-09-19T07:37:36.132 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 215328 2024-09-19T07:37:36.132 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.133 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2024-09-19T07:37:36.133 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 0, 2024-09-19T07:37:36.133 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 0, 2024-09-19T07:37:36.133 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 0, 2024-09-19T07:37:36.133 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 0, 2024-09-19T07:37:36.133 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 0, 2024-09-19T07:37:36.133 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2024-09-19T07:37:36.133 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.134 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.134 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.134 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.134 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2024-09-19T07:37:36.134 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.134 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.134 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.134 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.134 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2024-09-19T07:37:36.134 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.134 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.134 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.135 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.135 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2024-09-19T07:37:36.135 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.135 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.135 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.135 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.135 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2024-09-19T07:37:36.135 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.135 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.135 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.135 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.135 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2024-09-19T07:37:36.135 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.136 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.136 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.136 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.136 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2024-09-19T07:37:36.136 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.136 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.136 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.136 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.136 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2024-09-19T07:37:36.136 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.136 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.136 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.136 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.137 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2024-09-19T07:37:36.137 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.137 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.137 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.137 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.137 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2024-09-19T07:37:36.137 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.137 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.137 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.137 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.138 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2024-09-19T07:37:36.138 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.138 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.138 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.138 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.138 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2024-09-19T07:37:36.138 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.139 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.139 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.139 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.139 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2024-09-19T07:37:36.139 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.139 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.140 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.140 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.140 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2024-09-19T07:37:36.140 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.141 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.141 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.141 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.141 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2024-09-19T07:37:36.141 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.141 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.141 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.141 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.141 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2024-09-19T07:37:36.141 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.141 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.142 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.142 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.142 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2024-09-19T07:37:36.142 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.142 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.142 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.142 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.142 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2024-09-19T07:37:36.142 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.142 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.143 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.143 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.143 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2024-09-19T07:37:36.143 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.143 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.143 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.143 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.144 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2024-09-19T07:37:36.144 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.144 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.144 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.144 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.144 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2024-09-19T07:37:36.144 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.144 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.144 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.144 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.144 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2024-09-19T07:37:36.145 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.145 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.145 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.145 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.145 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2024-09-19T07:37:36.145 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.145 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.146 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.146 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.146 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2024-09-19T07:37:36.146 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.146 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.146 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.146 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.146 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2024-09-19T07:37:36.146 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.146 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.146 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.147 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.147 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2024-09-19T07:37:36.147 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.147 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.147 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.147 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.147 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2024-09-19T07:37:36.147 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.147 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.147 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.147 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.147 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2024-09-19T07:37:36.148 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.148 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.148 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.148 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.148 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2024-09-19T07:37:36.148 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.148 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.148 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.148 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.148 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:37:36.148 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:37:36.148 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2024-09-19T07:37:36.148 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.149 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.149 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.149 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:36.149 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.149 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2024-09-19T07:37:36.149 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 1, 2024-09-19T07:37:36.149 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 3, 2024-09-19T07:37:36.149 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 2, 2024-09-19T07:37:36.149 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 1, 2024-09-19T07:37:36.149 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2024-09-19T07:37:36.149 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 0, 2024-09-19T07:37:36.149 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 0, 2024-09-19T07:37:36.149 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 1788, 2024-09-19T07:37:36.150 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2024-09-19T07:37:36.150 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.150 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2024-09-19T07:37:36.150 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:37:36.150 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2024-09-19T07:37:36.150 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:37:36.150 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:37:36.150 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:37:36.150 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:37:36.150 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:37:36.150 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:37:36.150 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:37:36.150 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:37:36.150 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:37:36.151 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:37:36.151 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:37:36.151 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:37:36.151 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:37:36.151 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2024-09-19T07:37:36.151 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:37:36.151 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2024-09-19T07:37:36.151 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:37:36.151 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:37:36.151 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:37:36.151 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:37:36.151 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:37:36.151 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:37:36.152 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:37:36.152 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2024-09-19T07:37:36.152 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:37:36.152 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:37:36.152 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:37:36.152 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2024-09-19T07:37:36.152 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2024-09-19T07:37:36.152 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2024-09-19T07:37:36.152 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:37:36.152 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:37:36.152 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:37:36.152 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:37:36.152 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 189279, 2024-09-19T07:37:36.153 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 49, 2024-09-19T07:37:36.153 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:37:36.153 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2024-09-19T07:37:36.153 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2024-09-19T07:37:36.153 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2024-09-19T07:37:36.153 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:37:36.153 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2024-09-19T07:37:36.153 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:37:36.153 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2024-09-19T07:37:36.153 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 8952, 2024-09-19T07:37:36.153 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 168, 2024-09-19T07:37:36.153 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:37:36.153 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:37:36.153 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2024-09-19T07:37:36.154 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2024-09-19T07:37:36.154 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 56386, 2024-09-19T07:37:36.154 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 269, 2024-09-19T07:37:36.154 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2024-09-19T07:37:36.154 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2024-09-19T07:37:36.154 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2024-09-19T07:37:36.154 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2024-09-19T07:37:36.154 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.154 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2024-09-19T07:37:36.154 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2024-09-19T07:37:36.154 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2024-09-19T07:37:36.154 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 1114, 2024-09-19T07:37:36.154 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 26, 2024-09-19T07:37:36.154 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 4, 2024-09-19T07:37:36.155 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 1114, 2024-09-19T07:37:36.155 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2024-09-19T07:37:36.155 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1114, 2024-09-19T07:37:36.155 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.489987575, 2024-09-19T07:37:36.155 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000439845 2024-09-19T07:37:36.155 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.155 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2024-09-19T07:37:36.155 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2024-09-19T07:37:36.155 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1114, 2024-09-19T07:37:36.155 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 1115 2024-09-19T07:37:36.155 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.155 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 1114, 2024-09-19T07:37:36.155 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 1114, 2024-09-19T07:37:36.156 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 0, 2024-09-19T07:37:36.156 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2024-09-19T07:37:36.156 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2024-09-19T07:37:36.156 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 688, 2024-09-19T07:37:36.156 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 0, 2024-09-19T07:37:36.156 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 423, 2024-09-19T07:37:36.156 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 0, 2024-09-19T07:37:36.156 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 0, 2024-09-19T07:37:36.156 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2024-09-19T07:37:36.156 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2024-09-19T07:37:36.156 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2024-09-19T07:37:36.156 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 0, 2024-09-19T07:37:36.156 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 0, 2024-09-19T07:37:36.156 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2024-09-19T07:37:36.157 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2024-09-19T07:37:36.157 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2024-09-19T07:37:36.157 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 2, 2024-09-19T07:37:36.157 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 0, 2024-09-19T07:37:36.157 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:37:36.157 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2024-09-19T07:37:36.157 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:37:36.157 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2024-09-19T07:37:36.157 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2024-09-19T07:37:36.157 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2024-09-19T07:37:36.157 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:37:36.157 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2024-09-19T07:37:36.157 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:37:36.158 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 0, 2024-09-19T07:37:36.158 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2024-09-19T07:37:36.158 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2024-09-19T07:37:36.158 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2024-09-19T07:37:36.158 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2024-09-19T07:37:36.158 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2024-09-19T07:37:36.158 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2024-09-19T07:37:36.158 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2024-09-19T07:37:36.158 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2024-09-19T07:37:36.158 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2024-09-19T07:37:36.158 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2024-09-19T07:37:36.158 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2024-09-19T07:37:36.158 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2024-09-19T07:37:36.159 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2024-09-19T07:37:36.159 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2024-09-19T07:37:36.159 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2024-09-19T07:37:36.159 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2024-09-19T07:37:36.159 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 100, 2024-09-19T07:37:36.159 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2024-09-19T07:37:36.159 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 2, 2024-09-19T07:37:36.159 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 5, 2024-09-19T07:37:36.159 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 5, 2024-09-19T07:37:36.159 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2024-09-19T07:37:36.159 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2024-09-19T07:37:36.159 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 0, 2024-09-19T07:37:36.159 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 2, 2024-09-19T07:37:36.160 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 0 2024-09-19T07:37:36.160 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.160 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2024-09-19T07:37:36.160 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 0, 2024-09-19T07:37:36.160 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:37:36.160 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2024-09-19T07:37:36.160 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2024-09-19T07:37:36.160 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.160 INFO:teuthology.orchestra.run.smithi203.stdout: "osd-slow-ops": { 2024-09-19T07:37:36.160 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 0 2024-09-19T07:37:36.160 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.160 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2024-09-19T07:37:36.160 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2024-09-19T07:37:36.161 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:37:36.161 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2024-09-19T07:37:36.161 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 0, 2024-09-19T07:37:36.161 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed_ops": 0, 2024-09-19T07:37:36.161 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 0, 2024-09-19T07:37:36.161 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2024-09-19T07:37:36.161 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.161 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:37:36.161 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-19T07:37:36.161 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-09-19T07:37:36.161 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-19T07:37:36.161 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 1243, 2024-09-19T07:37:36.161 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 1344696, 2024-09-19T07:37:36.161 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:36.162 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 1243, 2024-09-19T07:37:36.162 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-19T07:37:36.162 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-19T07:37:36.162 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 1243, 2024-09-19T07:37:36.162 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 1344696, 2024-09-19T07:37:36.162 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-19T07:37:36.162 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.162 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.162 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.162 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:36.162 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.162 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2024-09-19T07:37:36.162 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-19T07:37:36.162 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-09-19T07:37:36.163 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-19T07:37:36.163 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-19T07:37:36.163 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-19T07:37:36.163 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:36.163 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:36.163 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 1114, 2024-09-19T07:37:36.163 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 676375, 2024-09-19T07:37:36.163 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 1114, 2024-09-19T07:37:36.163 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 676375, 2024-09-19T07:37:36.163 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-19T07:37:36.163 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.164 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.164 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.164 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:36.164 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.164 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2024-09-19T07:37:36.164 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-19T07:37:36.164 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2024-09-19T07:37:36.164 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-19T07:37:36.164 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-19T07:37:36.164 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-19T07:37:36.164 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:36.164 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:36.164 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 1114, 2024-09-19T07:37:36.165 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 1114, 2024-09-19T07:37:36.165 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 1114, 2024-09-19T07:37:36.165 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 1114, 2024-09-19T07:37:36.165 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-19T07:37:36.165 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.165 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.165 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.165 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:36.165 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.165 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:37:36.165 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-19T07:37:36.165 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-09-19T07:37:36.165 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-19T07:37:36.165 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-19T07:37:36.166 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-19T07:37:36.166 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:36.166 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:36.166 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-19T07:37:36.166 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-19T07:37:36.166 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2024-09-19T07:37:36.166 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2024-09-19T07:37:36.166 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-19T07:37:36.166 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.166 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.166 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.166 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:36.166 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:36.166 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x564190676b40": { 2024-09-19T07:37:36.167 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-19T07:37:36.167 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-09-19T07:37:36.167 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-19T07:37:36.167 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-19T07:37:36.167 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-19T07:37:36.167 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:36.167 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:36.167 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-19T07:37:36.167 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-19T07:37:36.167 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2024-09-19T07:37:36.167 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2024-09-19T07:37:36.167 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-19T07:37:36.167 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:36.168 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:36.168 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:36.168 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:36.168 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:36.168 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-09-19T07:37:36.272 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:35 smithi203 ceph-mon[35335]: pgmap v512: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-19T07:37:36.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:35 smithi189 ceph-mon[34851]: pgmap v512: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-19T07:37:36.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:35 smithi159 ceph-mon[30472]: pgmap v512: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-19T07:37:36.911 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-19T07:37:36.911 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-19T07:37:36.911 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-19T07:37:36.911 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-19T07:37:36.912 DEBUG:tasks.check_counter:Getting stats from f 2024-09-19T07:37:36.912 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-19T07:37:37.170 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:37:37.704 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-09-19T07:37:37.704 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:37:37.705 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 945, 2024-09-19T07:37:37.705 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 947, 2024-09-19T07:37:37.705 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 271202, 2024-09-19T07:37:37.705 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 300517, 2024-09-19T07:37:37.705 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 6, 2024-09-19T07:37:37.705 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 2, 2024-09-19T07:37:37.706 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.083184537, 2024-09-19T07:37:37.706 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.027245129, 2024-09-19T07:37:37.706 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.021610617, 2024-09-19T07:37:37.706 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.027340394, 2024-09-19T07:37:37.706 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:37.706 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 945, 2024-09-19T07:37:37.707 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.034895841, 2024-09-19T07:37:37.707 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000036926 2024-09-19T07:37:37.707 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.707 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:37.707 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.707 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.707 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.707 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.707 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:37:37.708 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:37:37.708 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.708 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:37:37.708 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 3, 2024-09-19T07:37:37.708 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 91, 2024-09-19T07:37:37.708 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 716, 2024-09-19T07:37:37.708 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 215674, 2024-09-19T07:37:37.708 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 4, 2024-09-19T07:37:37.708 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 3, 2024-09-19T07:37:37.708 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.005825606, 2024-09-19T07:37:37.708 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.003627002, 2024-09-19T07:37:37.709 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.001120488, 2024-09-19T07:37:37.709 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000228878, 2024-09-19T07:37:37.709 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:37.709 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 89, 2024-09-19T07:37:37.709 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.005336487, 2024-09-19T07:37:37.709 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000059960 2024-09-19T07:37:37.709 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.709 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:37.709 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.709 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.710 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.710 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.710 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-19T07:37:37.710 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 214976 2024-09-19T07:37:37.710 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.710 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:37:37.710 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 134, 2024-09-19T07:37:37.710 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 127, 2024-09-19T07:37:37.710 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 478460, 2024-09-19T07:37:37.710 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 73038, 2024-09-19T07:37:37.710 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 3, 2024-09-19T07:37:37.710 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 2, 2024-09-19T07:37:37.711 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.021463329, 2024-09-19T07:37:37.711 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.007928701, 2024-09-19T07:37:37.711 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.017558431, 2024-09-19T07:37:37.711 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000038125, 2024-09-19T07:37:37.711 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:37.711 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 127, 2024-09-19T07:37:37.711 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.011784483, 2024-09-19T07:37:37.711 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000092791 2024-09-19T07:37:37.711 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.711 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:37.711 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.711 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.711 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.711 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.712 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 478192, 2024-09-19T07:37:37.712 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 72720 2024-09-19T07:37:37.712 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.712 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2024-09-19T07:37:37.712 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2024-09-19T07:37:37.712 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2024-09-19T07:37:37.712 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.712 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2024-09-19T07:37:37.712 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-09-19T07:37:37.712 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-09-19T07:37:37.712 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1642, 2024-09-19T07:37:37.713 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.043342339, 2024-09-19T07:37:37.713 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000026396 2024-09-19T07:37:37.713 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:37.713 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.713 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2024-09-19T07:37:37.713 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-09-19T07:37:37.713 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-09-19T07:37:37.713 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.713 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.713 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.713 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:37.714 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.714 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2024-09-19T07:37:37.714 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 0, 2024-09-19T07:37:37.714 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 0, 2024-09-19T07:37:37.714 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2024-09-19T07:37:37.714 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.714 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.714 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.715 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.715 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2024-09-19T07:37:37.715 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 0, 2024-09-19T07:37:37.715 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 0, 2024-09-19T07:37:37.715 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2024-09-19T07:37:37.715 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 0, 2024-09-19T07:37:37.715 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 0, 2024-09-19T07:37:37.715 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 0, 2024-09-19T07:37:37.716 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 0, 2024-09-19T07:37:37.716 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 0, 2024-09-19T07:37:37.716 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 0, 2024-09-19T07:37:37.716 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 0, 2024-09-19T07:37:37.716 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 0, 2024-09-19T07:37:37.716 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 0, 2024-09-19T07:37:37.716 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 0, 2024-09-19T07:37:37.716 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 0, 2024-09-19T07:37:37.716 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 1, 2024-09-19T07:37:37.716 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 0, 2024-09-19T07:37:37.716 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 0, 2024-09-19T07:37:37.716 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 0, 2024-09-19T07:37:37.716 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 0, 2024-09-19T07:37:37.717 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:37:37.717 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2024-09-19T07:37:37.717 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 0, 2024-09-19T07:37:37.717 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 0, 2024-09-19T07:37:37.717 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2024-09-19T07:37:37.717 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 0, 2024-09-19T07:37:37.717 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 0, 2024-09-19T07:37:37.717 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 0, 2024-09-19T07:37:37.717 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 0, 2024-09-19T07:37:37.717 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2024-09-19T07:37:37.717 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:37:37.717 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 0, 2024-09-19T07:37:37.717 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 0, 2024-09-19T07:37:37.717 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 0, 2024-09-19T07:37:37.718 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 0, 2024-09-19T07:37:37.718 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:37:37.718 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2024-09-19T07:37:37.718 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:37:37.718 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:37:37.718 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:37:37.718 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:37:37.718 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:37:37.718 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 0, 2024-09-19T07:37:37.718 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:37:37.718 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T07:37:37.718 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:37:37.718 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:37:37.719 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:37:37.719 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:37:37.719 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 0, 2024-09-19T07:37:37.719 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:37:37.719 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 0, 2024-09-19T07:37:37.719 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 0 2024-09-19T07:37:37.719 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.720 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2024-09-19T07:37:37.720 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 0, 2024-09-19T07:37:37.720 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 0, 2024-09-19T07:37:37.720 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 0, 2024-09-19T07:37:37.720 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 0, 2024-09-19T07:37:37.720 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 0, 2024-09-19T07:37:37.720 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 0, 2024-09-19T07:37:37.720 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2024-09-19T07:37:37.720 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:37:37.720 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 0, 2024-09-19T07:37:37.720 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 0, 2024-09-19T07:37:37.720 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 0, 2024-09-19T07:37:37.721 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2024-09-19T07:37:37.721 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2024-09-19T07:37:37.721 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:37:37.721 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:37:37.721 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2024-09-19T07:37:37.721 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2024-09-19T07:37:37.721 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_path": 0, 2024-09-19T07:37:37.721 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_inode": 0, 2024-09-19T07:37:37.721 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:37:37.721 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 0, 2024-09-19T07:37:37.721 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2024-09-19T07:37:37.721 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:37:37.722 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2024-09-19T07:37:37.722 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0 2024-09-19T07:37:37.722 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.722 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2024-09-19T07:37:37.722 INFO:teuthology.orchestra.run.smithi203.stdout: "evlrg": 0, 2024-09-19T07:37:37.722 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 0, 2024-09-19T07:37:37.722 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 0, 2024-09-19T07:37:37.722 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 0, 2024-09-19T07:37:37.722 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 5, 2024-09-19T07:37:37.722 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2024-09-19T07:37:37.722 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 0, 2024-09-19T07:37:37.723 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 0, 2024-09-19T07:37:37.723 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 0, 2024-09-19T07:37:37.723 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 0, 2024-09-19T07:37:37.723 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 2, 2024-09-19T07:37:37.723 INFO:teuthology.orchestra.run.smithi203.stdout: "segmjr": 2, 2024-09-19T07:37:37.723 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2024-09-19T07:37:37.723 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 0, 2024-09-19T07:37:37.723 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 4194304, 2024-09-19T07:37:37.723 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 4195737, 2024-09-19T07:37:37.723 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 4195426, 2024-09-19T07:37:37.723 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2024-09-19T07:37:37.723 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.723 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.723 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.724 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.724 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 5 2024-09-19T07:37:37.724 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.724 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2024-09-19T07:37:37.724 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 3, 2024-09-19T07:37:37.724 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 3, 2024-09-19T07:37:37.724 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 0, 2024-09-19T07:37:37.724 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 1, 2024-09-19T07:37:37.724 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 1, 2024-09-19T07:37:37.724 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 0, 2024-09-19T07:37:37.724 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 0, 2024-09-19T07:37:37.724 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 0, 2024-09-19T07:37:37.724 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 0, 2024-09-19T07:37:37.724 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 0, 2024-09-19T07:37:37.725 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 0, 2024-09-19T07:37:37.725 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 0, 2024-09-19T07:37:37.725 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 50680, 2024-09-19T07:37:37.725 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 215328 2024-09-19T07:37:37.725 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.725 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2024-09-19T07:37:37.725 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 0, 2024-09-19T07:37:37.725 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 0, 2024-09-19T07:37:37.725 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 0, 2024-09-19T07:37:37.725 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 0, 2024-09-19T07:37:37.725 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 0, 2024-09-19T07:37:37.725 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2024-09-19T07:37:37.725 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.725 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.726 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.726 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.726 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2024-09-19T07:37:37.726 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.726 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.726 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.726 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.726 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2024-09-19T07:37:37.726 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.726 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.726 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.726 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.726 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2024-09-19T07:37:37.726 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.727 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.727 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.727 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.727 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2024-09-19T07:37:37.727 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.727 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.727 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.727 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.727 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2024-09-19T07:37:37.727 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.727 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.727 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.728 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.728 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2024-09-19T07:37:37.728 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.728 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.728 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.728 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.728 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2024-09-19T07:37:37.728 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.728 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.728 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.729 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.729 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2024-09-19T07:37:37.729 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.729 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.729 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.729 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.729 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2024-09-19T07:37:37.729 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.729 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.729 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.729 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.729 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2024-09-19T07:37:37.729 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.729 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.730 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.730 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.730 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2024-09-19T07:37:37.730 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.730 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.730 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.730 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.730 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2024-09-19T07:37:37.730 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.730 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.730 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.730 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.730 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2024-09-19T07:37:37.730 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.731 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.731 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.731 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.731 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2024-09-19T07:37:37.731 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.731 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.731 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.731 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.731 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2024-09-19T07:37:37.731 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.731 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.731 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.731 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.731 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2024-09-19T07:37:37.732 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.732 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.732 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.732 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.732 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2024-09-19T07:37:37.732 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.732 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.732 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.732 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.732 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2024-09-19T07:37:37.732 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.732 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.732 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.732 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.733 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2024-09-19T07:37:37.733 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.733 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.733 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.733 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.733 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2024-09-19T07:37:37.733 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.733 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.733 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.733 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.733 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2024-09-19T07:37:37.733 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.733 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.733 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.734 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.734 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2024-09-19T07:37:37.734 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.734 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.734 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.734 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.734 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2024-09-19T07:37:37.734 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.734 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.734 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.734 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.734 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2024-09-19T07:37:37.734 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.734 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.735 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.735 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.735 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2024-09-19T07:37:37.735 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.735 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.735 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.735 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.735 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2024-09-19T07:37:37.735 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.735 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.735 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.735 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.735 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2024-09-19T07:37:37.735 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.736 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.736 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.736 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.736 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2024-09-19T07:37:37.736 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.736 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.736 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.736 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.736 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:37:37.736 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:37:37.736 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2024-09-19T07:37:37.736 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.737 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.737 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.737 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:37.737 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.737 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2024-09-19T07:37:37.737 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 1, 2024-09-19T07:37:37.737 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 2, 2024-09-19T07:37:37.737 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 1, 2024-09-19T07:37:37.737 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 1, 2024-09-19T07:37:37.737 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2024-09-19T07:37:37.737 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 0, 2024-09-19T07:37:37.737 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 0, 2024-09-19T07:37:37.737 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 1791, 2024-09-19T07:37:37.738 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2024-09-19T07:37:37.738 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.738 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2024-09-19T07:37:37.738 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:37:37.738 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2024-09-19T07:37:37.738 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:37:37.738 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:37:37.738 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:37:37.738 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:37:37.738 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:37:37.738 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:37:37.738 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:37:37.738 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:37:37.738 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:37:37.739 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:37:37.739 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:37:37.739 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:37:37.739 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:37:37.739 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2024-09-19T07:37:37.739 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:37:37.739 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2024-09-19T07:37:37.739 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:37:37.739 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:37:37.739 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:37:37.739 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:37:37.739 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:37:37.739 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:37:37.739 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:37:37.739 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2024-09-19T07:37:37.740 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:37:37.740 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:37:37.740 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:37:37.740 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2024-09-19T07:37:37.740 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2024-09-19T07:37:37.740 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2024-09-19T07:37:37.740 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:37:37.740 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:37:37.740 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:37:37.740 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:37:37.740 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 181199, 2024-09-19T07:37:37.740 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 45, 2024-09-19T07:37:37.740 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:37:37.741 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2024-09-19T07:37:37.741 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2024-09-19T07:37:37.741 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2024-09-19T07:37:37.741 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:37:37.741 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2024-09-19T07:37:37.741 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:37:37.741 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2024-09-19T07:37:37.741 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 8952, 2024-09-19T07:37:37.741 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 168, 2024-09-19T07:37:37.741 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:37:37.741 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:37:37.741 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2024-09-19T07:37:37.741 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2024-09-19T07:37:37.741 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 8512, 2024-09-19T07:37:37.742 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 13, 2024-09-19T07:37:37.742 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2024-09-19T07:37:37.742 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2024-09-19T07:37:37.742 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2024-09-19T07:37:37.742 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2024-09-19T07:37:37.742 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.742 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2024-09-19T07:37:37.742 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2024-09-19T07:37:37.742 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2024-09-19T07:37:37.742 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 948, 2024-09-19T07:37:37.742 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 21, 2024-09-19T07:37:37.742 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 3, 2024-09-19T07:37:37.742 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 948, 2024-09-19T07:37:37.742 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2024-09-19T07:37:37.743 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 948, 2024-09-19T07:37:37.743 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.467988141, 2024-09-19T07:37:37.743 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000493658 2024-09-19T07:37:37.743 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.743 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2024-09-19T07:37:37.743 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2024-09-19T07:37:37.743 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 948, 2024-09-19T07:37:37.743 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 949 2024-09-19T07:37:37.743 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.743 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 948, 2024-09-19T07:37:37.744 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 948, 2024-09-19T07:37:37.744 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 0, 2024-09-19T07:37:37.744 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2024-09-19T07:37:37.744 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2024-09-19T07:37:37.744 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 593, 2024-09-19T07:37:37.744 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 0, 2024-09-19T07:37:37.744 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 353, 2024-09-19T07:37:37.744 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 0, 2024-09-19T07:37:37.745 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 0, 2024-09-19T07:37:37.745 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2024-09-19T07:37:37.745 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2024-09-19T07:37:37.745 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2024-09-19T07:37:37.745 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 0, 2024-09-19T07:37:37.745 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 0, 2024-09-19T07:37:37.745 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2024-09-19T07:37:37.745 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2024-09-19T07:37:37.745 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2024-09-19T07:37:37.745 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 1, 2024-09-19T07:37:37.745 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 0, 2024-09-19T07:37:37.745 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:37:37.746 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2024-09-19T07:37:37.746 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:37:37.746 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2024-09-19T07:37:37.746 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2024-09-19T07:37:37.746 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2024-09-19T07:37:37.746 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:37:37.746 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2024-09-19T07:37:37.746 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:37:37.747 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 0, 2024-09-19T07:37:37.747 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2024-09-19T07:37:37.747 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2024-09-19T07:37:37.747 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2024-09-19T07:37:37.747 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2024-09-19T07:37:37.747 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2024-09-19T07:37:37.748 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2024-09-19T07:37:37.748 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2024-09-19T07:37:37.748 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2024-09-19T07:37:37.748 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2024-09-19T07:37:37.748 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2024-09-19T07:37:37.748 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2024-09-19T07:37:37.748 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2024-09-19T07:37:37.748 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2024-09-19T07:37:37.748 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2024-09-19T07:37:37.748 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2024-09-19T07:37:37.748 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2024-09-19T07:37:37.749 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 100, 2024-09-19T07:37:37.749 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2024-09-19T07:37:37.749 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 2, 2024-09-19T07:37:37.749 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 4, 2024-09-19T07:37:37.749 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 4, 2024-09-19T07:37:37.749 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2024-09-19T07:37:37.749 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2024-09-19T07:37:37.749 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 0, 2024-09-19T07:37:37.749 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 2, 2024-09-19T07:37:37.749 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 0 2024-09-19T07:37:37.749 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.749 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2024-09-19T07:37:37.749 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 0, 2024-09-19T07:37:37.749 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:37:37.750 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2024-09-19T07:37:37.750 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2024-09-19T07:37:37.750 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.750 INFO:teuthology.orchestra.run.smithi203.stdout: "osd-slow-ops": { 2024-09-19T07:37:37.750 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 0 2024-09-19T07:37:37.750 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.750 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2024-09-19T07:37:37.750 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2024-09-19T07:37:37.750 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:37:37.750 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2024-09-19T07:37:37.750 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 0, 2024-09-19T07:37:37.750 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed_ops": 0, 2024-09-19T07:37:37.750 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 0, 2024-09-19T07:37:37.750 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2024-09-19T07:37:37.750 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.751 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:37:37.751 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-19T07:37:37.751 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-09-19T07:37:37.751 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-19T07:37:37.751 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 1078, 2024-09-19T07:37:37.751 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 643725, 2024-09-19T07:37:37.751 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:37.751 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 1078, 2024-09-19T07:37:37.751 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-19T07:37:37.751 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-19T07:37:37.752 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 1078, 2024-09-19T07:37:37.752 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 643725, 2024-09-19T07:37:37.752 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-19T07:37:37.752 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.752 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.752 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.752 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:37.752 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.752 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2024-09-19T07:37:37.752 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-19T07:37:37.752 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-09-19T07:37:37.752 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-19T07:37:37.752 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-19T07:37:37.752 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-19T07:37:37.753 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:37.753 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:37.753 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 948, 2024-09-19T07:37:37.753 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 1438, 2024-09-19T07:37:37.753 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 948, 2024-09-19T07:37:37.753 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 1438, 2024-09-19T07:37:37.753 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-19T07:37:37.753 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.753 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.754 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.754 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:37.754 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.754 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2024-09-19T07:37:37.754 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-19T07:37:37.754 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2024-09-19T07:37:37.755 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-19T07:37:37.755 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-19T07:37:37.755 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-19T07:37:37.755 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:37.755 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:37.755 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 948, 2024-09-19T07:37:37.755 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 948, 2024-09-19T07:37:37.755 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 948, 2024-09-19T07:37:37.755 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 948, 2024-09-19T07:37:37.755 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-19T07:37:37.755 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.755 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.755 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.755 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:37.755 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.756 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:37:37.756 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-19T07:37:37.756 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-09-19T07:37:37.756 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-19T07:37:37.756 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-19T07:37:37.756 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-19T07:37:37.756 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:37.756 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:37.756 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-19T07:37:37.756 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-19T07:37:37.756 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2024-09-19T07:37:37.756 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2024-09-19T07:37:37.756 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-19T07:37:37.756 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.757 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.757 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.757 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:37.757 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:37.757 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x555ce4144b40": { 2024-09-19T07:37:37.757 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-19T07:37:37.757 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-09-19T07:37:37.757 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-19T07:37:37.757 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-19T07:37:37.757 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-19T07:37:37.757 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:37.757 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:37.757 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-19T07:37:37.757 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-19T07:37:37.758 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2024-09-19T07:37:37.758 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2024-09-19T07:37:37.758 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-19T07:37:37.758 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:37.758 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:37.758 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:37.758 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:37.758 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:37.758 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-09-19T07:37:38.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:38 smithi203 ceph-mon[35335]: pgmap v513: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-19T07:37:38.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:38 smithi189 ceph-mon[34851]: pgmap v513: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-19T07:37:38.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:38 smithi159 ceph-mon[30472]: pgmap v513: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-19T07:37:38.658 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-19T07:37:38.658 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-19T07:37:38.658 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-19T07:37:38.658 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-19T07:37:38.658 DEBUG:tasks.check_counter:Getting stats from i 2024-09-19T07:37:38.659 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-19T07:37:38.906 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:37:39.412 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-09-19T07:37:39.412 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:37:39.412 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 178, 2024-09-19T07:37:39.413 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 252, 2024-09-19T07:37:39.413 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 50395, 2024-09-19T07:37:39.413 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 106584, 2024-09-19T07:37:39.413 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 9, 2024-09-19T07:37:39.413 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 3, 2024-09-19T07:37:39.413 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.029519332, 2024-09-19T07:37:39.413 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.009458977, 2024-09-19T07:37:39.414 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.015326495, 2024-09-19T07:37:39.414 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.002341543, 2024-09-19T07:37:39.414 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:39.414 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 251, 2024-09-19T07:37:39.414 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.041323113, 2024-09-19T07:37:39.414 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000164633 2024-09-19T07:37:39.414 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.415 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:39.415 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 375, 2024-09-19T07:37:39.415 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000117875, 2024-09-19T07:37:39.415 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000314 2024-09-19T07:37:39.415 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.415 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-19T07:37:39.415 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T07:37:39.415 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.415 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:37:39.416 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 22, 2024-09-19T07:37:39.416 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 109, 2024-09-19T07:37:39.416 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 5640, 2024-09-19T07:37:39.416 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 247827, 2024-09-19T07:37:39.416 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 10, 2024-09-19T07:37:39.416 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 7, 2024-09-19T07:37:39.416 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.010289460, 2024-09-19T07:37:39.416 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.004752400, 2024-09-19T07:37:39.416 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.004229943, 2024-09-19T07:37:39.416 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000264203, 2024-09-19T07:37:39.416 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:39.416 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 107, 2024-09-19T07:37:39.417 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.013481788, 2024-09-19T07:37:39.417 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000125998 2024-09-19T07:37:39.417 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.417 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:39.417 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 14, 2024-09-19T07:37:39.417 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000004283, 2024-09-19T07:37:39.417 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000305 2024-09-19T07:37:39.417 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.417 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-09-19T07:37:39.417 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 217152 2024-09-19T07:37:39.418 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.418 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:37:39.418 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 186, 2024-09-19T07:37:39.418 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 177, 2024-09-19T07:37:39.418 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 500052, 2024-09-19T07:37:39.418 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 119010, 2024-09-19T07:37:39.418 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 8, 2024-09-19T07:37:39.418 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 5, 2024-09-19T07:37:39.418 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.024550812, 2024-09-19T07:37:39.418 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.009262364, 2024-09-19T07:37:39.418 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.018794425, 2024-09-19T07:37:39.419 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000172701, 2024-09-19T07:37:39.419 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:39.420 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 176, 2024-09-19T07:37:39.420 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.017620807, 2024-09-19T07:37:39.420 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000100118 2024-09-19T07:37:39.420 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.420 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:39.420 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 40, 2024-09-19T07:37:39.420 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000007943, 2024-09-19T07:37:39.420 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000198 2024-09-19T07:37:39.421 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.421 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 487760, 2024-09-19T07:37:39.421 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 75472 2024-09-19T07:37:39.421 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.421 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2024-09-19T07:37:39.421 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2024-09-19T07:37:39.421 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2024-09-19T07:37:39.421 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.421 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2024-09-19T07:37:39.422 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-09-19T07:37:39.422 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-09-19T07:37:39.422 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 45, 2024-09-19T07:37:39.422 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.001810039, 2024-09-19T07:37:39.422 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000040223 2024-09-19T07:37:39.422 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:39.423 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.423 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2024-09-19T07:37:39.423 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-09-19T07:37:39.423 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-09-19T07:37:39.424 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2, 2024-09-19T07:37:39.424 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000012362, 2024-09-19T07:37:39.424 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000006181 2024-09-19T07:37:39.424 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:39.424 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.424 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2024-09-19T07:37:39.425 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 2, 2024-09-19T07:37:39.425 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 1, 2024-09-19T07:37:39.425 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2024-09-19T07:37:39.425 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1, 2024-09-19T07:37:39.425 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000179440, 2024-09-19T07:37:39.425 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000179440 2024-09-19T07:37:39.426 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.426 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2024-09-19T07:37:39.426 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 1, 2024-09-19T07:37:39.426 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 0, 2024-09-19T07:37:39.426 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2024-09-19T07:37:39.426 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 11, 2024-09-19T07:37:39.426 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 0, 2024-09-19T07:37:39.427 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 0, 2024-09-19T07:37:39.427 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 14, 2024-09-19T07:37:39.427 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 1, 2024-09-19T07:37:39.427 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 13, 2024-09-19T07:37:39.427 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 0, 2024-09-19T07:37:39.427 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 12, 2024-09-19T07:37:39.428 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 0, 2024-09-19T07:37:39.428 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 1, 2024-09-19T07:37:39.428 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 1, 2024-09-19T07:37:39.428 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 2, 2024-09-19T07:37:39.428 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 6, 2024-09-19T07:37:39.428 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 3, 2024-09-19T07:37:39.429 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 1, 2024-09-19T07:37:39.429 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 2, 2024-09-19T07:37:39.429 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:37:39.429 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2024-09-19T07:37:39.429 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 0, 2024-09-19T07:37:39.429 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 0, 2024-09-19T07:37:39.430 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2024-09-19T07:37:39.430 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 0, 2024-09-19T07:37:39.430 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 0, 2024-09-19T07:37:39.430 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 0, 2024-09-19T07:37:39.430 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 0, 2024-09-19T07:37:39.430 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2024-09-19T07:37:39.431 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:37:39.431 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 0, 2024-09-19T07:37:39.431 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 3, 2024-09-19T07:37:39.431 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 117, 2024-09-19T07:37:39.431 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 0, 2024-09-19T07:37:39.431 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:37:39.432 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2024-09-19T07:37:39.432 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:37:39.432 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:37:39.432 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:37:39.432 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:37:39.432 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:37:39.433 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 0, 2024-09-19T07:37:39.433 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:37:39.433 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T07:37:39.433 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:37:39.433 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:37:39.433 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:37:39.434 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:37:39.434 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 0, 2024-09-19T07:37:39.434 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:37:39.434 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 0, 2024-09-19T07:37:39.434 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 0 2024-09-19T07:37:39.434 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.435 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2024-09-19T07:37:39.435 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 0, 2024-09-19T07:37:39.435 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 8, 2024-09-19T07:37:39.435 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 5, 2024-09-19T07:37:39.435 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 6, 2024-09-19T07:37:39.435 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 0, 2024-09-19T07:37:39.436 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 0, 2024-09-19T07:37:39.436 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2024-09-19T07:37:39.436 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:37:39.436 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 0, 2024-09-19T07:37:39.436 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 0, 2024-09-19T07:37:39.436 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 0, 2024-09-19T07:37:39.437 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2024-09-19T07:37:39.437 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2024-09-19T07:37:39.437 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:37:39.437 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:37:39.437 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2024-09-19T07:37:39.437 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2024-09-19T07:37:39.438 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_path": 0, 2024-09-19T07:37:39.438 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_inode": 0, 2024-09-19T07:37:39.438 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:37:39.438 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 0, 2024-09-19T07:37:39.438 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2024-09-19T07:37:39.438 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:37:39.438 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2024-09-19T07:37:39.438 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0 2024-09-19T07:37:39.439 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.439 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2024-09-19T07:37:39.439 INFO:teuthology.orchestra.run.smithi203.stdout: "evlrg": 0, 2024-09-19T07:37:39.439 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 5, 2024-09-19T07:37:39.439 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 0, 2024-09-19T07:37:39.439 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 0, 2024-09-19T07:37:39.440 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 5, 2024-09-19T07:37:39.440 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2024-09-19T07:37:39.440 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 0, 2024-09-19T07:37:39.440 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 2, 2024-09-19T07:37:39.440 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 0, 2024-09-19T07:37:39.440 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 0, 2024-09-19T07:37:39.440 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 2, 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi203.stdout: "segmjr": 2, 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 0, 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 4194304, 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 4195737, 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 0, 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 4, 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.006469589, 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001617397 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 0 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 17, 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 17, 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 0, 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 16, 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 18, 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 2, 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 14, 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 14, 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 0, 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 1, 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 1, 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 0, 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 49356, 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 223520 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 2, 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 0, 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 2, 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 7, 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 0, 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.445 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2024-09-19T07:37:39.445 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.445 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.445 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.445 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.445 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2024-09-19T07:37:39.445 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.445 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.445 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.445 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.445 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2024-09-19T07:37:39.446 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.446 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.446 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.446 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.446 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2024-09-19T07:37:39.446 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1, 2024-09-19T07:37:39.446 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000179440, 2024-09-19T07:37:39.446 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000179440 2024-09-19T07:37:39.446 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.446 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2024-09-19T07:37:39.446 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.447 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.447 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.447 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.447 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2024-09-19T07:37:39.447 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.447 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.447 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.447 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.447 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2024-09-19T07:37:39.447 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.447 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.447 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.447 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.447 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2024-09-19T07:37:39.448 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.448 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.448 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.448 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.448 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2024-09-19T07:37:39.448 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.449 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.449 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.449 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.449 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2024-09-19T07:37:39.449 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.449 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.449 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.449 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.449 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2024-09-19T07:37:39.449 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.450 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.450 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.450 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.450 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2024-09-19T07:37:39.450 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.450 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.450 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.450 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.450 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2024-09-19T07:37:39.450 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.450 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.450 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.450 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.450 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2024-09-19T07:37:39.451 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.451 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.451 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.451 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.451 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2024-09-19T07:37:39.451 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.451 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.451 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.451 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.451 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2024-09-19T07:37:39.451 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.452 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.452 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.452 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.452 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2024-09-19T07:37:39.452 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.452 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.452 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.452 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.452 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2024-09-19T07:37:39.452 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.452 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.452 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.452 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.453 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2024-09-19T07:37:39.453 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.453 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.453 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.453 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.453 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2024-09-19T07:37:39.453 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.453 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.457 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.457 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.457 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2024-09-19T07:37:39.457 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.457 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.458 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.458 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.458 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2024-09-19T07:37:39.458 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.458 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.458 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.458 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.458 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2024-09-19T07:37:39.458 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.458 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.459 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.459 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.459 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2024-09-19T07:37:39.459 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.459 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.459 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.460 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.460 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2024-09-19T07:37:39.460 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.460 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.460 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.460 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.460 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2024-09-19T07:37:39.460 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.460 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.461 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.461 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.461 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:37:39.461 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:37:39.461 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2024-09-19T07:37:39.461 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.461 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.461 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.461 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:39.462 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.462 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2024-09-19T07:37:39.462 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 1, 2024-09-19T07:37:39.462 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 2, 2024-09-19T07:37:39.462 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 1, 2024-09-19T07:37:39.462 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 1, 2024-09-19T07:37:39.462 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2024-09-19T07:37:39.462 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 1, 2024-09-19T07:37:39.462 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 1, 2024-09-19T07:37:39.462 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 229, 2024-09-19T07:37:39.462 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2024-09-19T07:37:39.462 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.463 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2024-09-19T07:37:39.463 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:37:39.463 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2024-09-19T07:37:39.463 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:37:39.463 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:37:39.463 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:37:39.463 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:37:39.463 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:37:39.463 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:37:39.463 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:37:39.463 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:37:39.463 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:37:39.464 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:37:39.464 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:37:39.464 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:37:39.464 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:37:39.464 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2024-09-19T07:37:39.464 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:37:39.464 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2024-09-19T07:37:39.464 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:37:39.464 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:37:39.464 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:37:39.464 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:37:39.464 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:37:39.464 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:37:39.465 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:37:39.465 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2024-09-19T07:37:39.465 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:37:39.465 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:37:39.465 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:37:39.465 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2024-09-19T07:37:39.465 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2024-09-19T07:37:39.465 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2024-09-19T07:37:39.465 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:37:39.465 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:37:39.465 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:37:39.466 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:37:39.466 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 247689, 2024-09-19T07:37:39.466 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 86, 2024-09-19T07:37:39.466 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:37:39.466 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2024-09-19T07:37:39.466 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2024-09-19T07:37:39.466 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2024-09-19T07:37:39.466 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:37:39.466 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2024-09-19T07:37:39.466 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:37:39.466 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2024-09-19T07:37:39.466 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 8952, 2024-09-19T07:37:39.466 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 168, 2024-09-19T07:37:39.466 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:37:39.467 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:37:39.467 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2024-09-19T07:37:39.467 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2024-09-19T07:37:39.467 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 69424, 2024-09-19T07:37:39.467 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 257, 2024-09-19T07:37:39.467 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2024-09-19T07:37:39.467 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2024-09-19T07:37:39.467 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2024-09-19T07:37:39.467 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2024-09-19T07:37:39.467 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.467 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2024-09-19T07:37:39.467 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2024-09-19T07:37:39.467 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2024-09-19T07:37:39.467 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 40, 2024-09-19T07:37:39.467 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 11735, 2024-09-19T07:37:39.468 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 28, 2024-09-19T07:37:39.468 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 40, 2024-09-19T07:37:39.468 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2024-09-19T07:37:39.468 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 40, 2024-09-19T07:37:39.468 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.228994206, 2024-09-19T07:37:39.468 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.005724855 2024-09-19T07:37:39.468 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.468 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2024-09-19T07:37:39.468 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2024-09-19T07:37:39.469 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 40, 2024-09-19T07:37:39.469 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 62 2024-09-19T07:37:39.469 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.469 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 40, 2024-09-19T07:37:39.469 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 0, 2024-09-19T07:37:39.469 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 40, 2024-09-19T07:37:39.469 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2024-09-19T07:37:39.469 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2024-09-19T07:37:39.469 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 0, 2024-09-19T07:37:39.469 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 10, 2024-09-19T07:37:39.469 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 0, 2024-09-19T07:37:39.469 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 4, 2024-09-19T07:37:39.469 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 7, 2024-09-19T07:37:39.469 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2024-09-19T07:37:39.470 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2024-09-19T07:37:39.470 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2024-09-19T07:37:39.470 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 0, 2024-09-19T07:37:39.470 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 6, 2024-09-19T07:37:39.470 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2024-09-19T07:37:39.470 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2024-09-19T07:37:39.470 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2024-09-19T07:37:39.470 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 0, 2024-09-19T07:37:39.470 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 20, 2024-09-19T07:37:39.470 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:37:39.470 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2024-09-19T07:37:39.470 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:37:39.470 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2024-09-19T07:37:39.470 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2024-09-19T07:37:39.471 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2024-09-19T07:37:39.471 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:37:39.471 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2024-09-19T07:37:39.471 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:37:39.471 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 0, 2024-09-19T07:37:39.471 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2024-09-19T07:37:39.471 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2024-09-19T07:37:39.471 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2024-09-19T07:37:39.471 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2024-09-19T07:37:39.471 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2024-09-19T07:37:39.471 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2024-09-19T07:37:39.471 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2024-09-19T07:37:39.472 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2024-09-19T07:37:39.472 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2024-09-19T07:37:39.472 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2024-09-19T07:37:39.472 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2024-09-19T07:37:39.472 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2024-09-19T07:37:39.472 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2024-09-19T07:37:39.472 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2024-09-19T07:37:39.472 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2024-09-19T07:37:39.472 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2024-09-19T07:37:39.473 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 100, 2024-09-19T07:37:39.473 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2024-09-19T07:37:39.473 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 2, 2024-09-19T07:37:39.473 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 9, 2024-09-19T07:37:39.473 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 9, 2024-09-19T07:37:39.473 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2024-09-19T07:37:39.473 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2024-09-19T07:37:39.473 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 14, 2024-09-19T07:37:39.473 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 0, 2024-09-19T07:37:39.474 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 1 2024-09-19T07:37:39.474 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.474 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2024-09-19T07:37:39.474 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 1, 2024-09-19T07:37:39.474 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:37:39.474 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2024-09-19T07:37:39.474 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2024-09-19T07:37:39.474 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.475 INFO:teuthology.orchestra.run.smithi203.stdout: "osd-slow-ops": { 2024-09-19T07:37:39.475 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 0 2024-09-19T07:37:39.475 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.475 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2024-09-19T07:37:39.475 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2024-09-19T07:37:39.475 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:37:39.475 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2024-09-19T07:37:39.475 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 0, 2024-09-19T07:37:39.475 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed_ops": 0, 2024-09-19T07:37:39.476 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 0, 2024-09-19T07:37:39.476 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2024-09-19T07:37:39.476 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.476 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:37:39.476 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-19T07:37:39.476 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-09-19T07:37:39.476 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-19T07:37:39.476 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 382, 2024-09-19T07:37:39.476 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 510752, 2024-09-19T07:37:39.477 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:39.477 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 382, 2024-09-19T07:37:39.477 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-19T07:37:39.477 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-19T07:37:39.477 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 382, 2024-09-19T07:37:39.477 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 510752, 2024-09-19T07:37:39.477 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-19T07:37:39.477 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.477 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.478 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.478 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:39.478 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.478 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2024-09-19T07:37:39.478 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-19T07:37:39.478 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-09-19T07:37:39.478 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-19T07:37:39.478 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-19T07:37:39.478 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-19T07:37:39.479 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:39.479 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:39.479 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 40, 2024-09-19T07:37:39.479 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 11735, 2024-09-19T07:37:39.479 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 40, 2024-09-19T07:37:39.479 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 11735, 2024-09-19T07:37:39.479 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-19T07:37:39.479 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.479 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.480 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.480 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:39.480 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.480 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2024-09-19T07:37:39.480 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-19T07:37:39.480 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2024-09-19T07:37:39.480 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-19T07:37:39.480 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-19T07:37:39.481 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-19T07:37:39.481 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:39.481 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:39.481 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 40, 2024-09-19T07:37:39.481 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 40, 2024-09-19T07:37:39.481 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 40, 2024-09-19T07:37:39.481 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 40, 2024-09-19T07:37:39.481 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-19T07:37:39.481 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.482 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.482 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.482 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:39.482 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.482 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:37:39.482 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-19T07:37:39.482 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-09-19T07:37:39.482 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-19T07:37:39.482 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-09-19T07:37:39.483 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-09-19T07:37:39.483 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:39.483 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-09-19T07:37:39.483 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-19T07:37:39.483 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-19T07:37:39.483 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2024-09-19T07:37:39.483 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2024-09-19T07:37:39.483 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-19T07:37:39.483 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.483 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.484 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.484 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:39.484 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:39.484 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x5607ca23a960": { 2024-09-19T07:37:39.484 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-19T07:37:39.484 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-09-19T07:37:39.484 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-19T07:37:39.484 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 5, 2024-09-19T07:37:39.484 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 1433, 2024-09-19T07:37:39.485 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:39.485 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 5, 2024-09-19T07:37:39.485 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-19T07:37:39.485 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-19T07:37:39.485 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 4, 2024-09-19T07:37:39.485 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 1433, 2024-09-19T07:37:39.485 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-19T07:37:39.485 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:39.485 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:39.486 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:39.486 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:39.486 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:39.486 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-09-19T07:37:40.207 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-19T07:37:40.207 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-19T07:37:40.207 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-19T07:37:40.207 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=8 2024-09-19T07:37:40.207 DEBUG:tasks.check_counter:Getting stats from l 2024-09-19T07:37:40.208 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-19T07:37:40.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:40 smithi203 ceph-mon[35335]: pgmap v514: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T07:37:40.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:40 smithi189 ceph-mon[34851]: pgmap v514: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T07:37:40.388 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:40 smithi159 ceph-mon[30472]: pgmap v514: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-19T07:37:40.455 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:37:41.025 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-09-19T07:37:41.025 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:37:41.025 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 0, 2024-09-19T07:37:41.025 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 2, 2024-09-19T07:37:41.025 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 0, 2024-09-19T07:37:41.025 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 0, 2024-09-19T07:37:41.025 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 4, 2024-09-19T07:37:41.026 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 0, 2024-09-19T07:37:41.026 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.001129656, 2024-09-19T07:37:41.026 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T07:37:41.026 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.000203590, 2024-09-19T07:37:41.026 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:41.026 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:41.026 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:41.026 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:41.026 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:41.026 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:41.026 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:41.026 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:41.026 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:41.027 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:41.027 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:41.027 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:37:41.027 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:37:41.027 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:41.027 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:37:41.027 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 5, 2024-09-19T07:37:41.027 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 95, 2024-09-19T07:37:41.027 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 3184, 2024-09-19T07:37:41.027 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 95344, 2024-09-19T07:37:41.028 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 3, 2024-09-19T07:37:41.028 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 2, 2024-09-19T07:37:41.028 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.007786232, 2024-09-19T07:37:41.028 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.004144018, 2024-09-19T07:37:41.028 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.002557299, 2024-09-19T07:37:41.028 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:41.028 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:41.028 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 94, 2024-09-19T07:37:41.028 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.009531400, 2024-09-19T07:37:41.028 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000101397 2024-09-19T07:37:41.028 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:41.028 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:41.028 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:41.029 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:41.029 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:41.029 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:41.029 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-19T07:37:41.029 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 95344 2024-09-19T07:37:41.029 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:41.029 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:37:41.029 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 125, 2024-09-19T07:37:41.029 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 122, 2024-09-19T07:37:41.029 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 452816, 2024-09-19T07:37:41.029 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 74288, 2024-09-19T07:37:41.030 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 2, 2024-09-19T07:37:41.030 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 1, 2024-09-19T07:37:41.030 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.018966834, 2024-09-19T07:37:41.030 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.008272810, 2024-09-19T07:37:41.030 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.014152471, 2024-09-19T07:37:41.030 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:37:41.030 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:37:41.030 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 121, 2024-09-19T07:37:41.030 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.007618905, 2024-09-19T07:37:41.030 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000062966 2024-09-19T07:37:41.030 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:41.030 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:37:41.031 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:41.031 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:41.031 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:41.031 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:41.031 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 452816, 2024-09-19T07:37:41.031 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 74288 2024-09-19T07:37:41.031 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:41.031 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2024-09-19T07:37:41.031 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 0, 2024-09-19T07:37:41.031 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2024-09-19T07:37:41.031 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:41.031 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2024-09-19T07:37:41.031 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:37:41.032 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2024-09-19T07:37:41.032 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:37:41.032 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:37:41.032 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:37:41.032 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:37:41.032 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:37:41.032 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:37:41.032 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:37:41.032 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:37:41.032 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:37:41.032 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:37:41.032 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:37:41.032 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:37:41.033 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:37:41.033 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2024-09-19T07:37:41.033 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:37:41.033 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2024-09-19T07:37:41.033 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:37:41.033 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:37:41.033 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:37:41.033 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:37:41.033 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:37:41.033 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:37:41.033 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:37:41.033 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2024-09-19T07:37:41.033 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:37:41.033 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:37:41.034 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:37:41.034 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2024-09-19T07:37:41.034 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2024-09-19T07:37:41.034 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2024-09-19T07:37:41.034 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:37:41.034 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:37:41.034 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:37:41.034 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:37:41.034 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 148801, 2024-09-19T07:37:41.034 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 27, 2024-09-19T07:37:41.034 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:37:41.034 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2024-09-19T07:37:41.034 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2024-09-19T07:37:41.034 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2024-09-19T07:37:41.034 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:37:41.035 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2024-09-19T07:37:41.035 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:37:41.035 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2024-09-19T07:37:41.035 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 0, 2024-09-19T07:37:41.035 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 0, 2024-09-19T07:37:41.035 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:37:41.035 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:37:41.035 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2024-09-19T07:37:41.035 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2024-09-19T07:37:41.035 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 928, 2024-09-19T07:37:41.036 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 2, 2024-09-19T07:37:41.036 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2024-09-19T07:37:41.036 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2024-09-19T07:37:41.036 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2024-09-19T07:37:41.036 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2024-09-19T07:37:41.036 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-09-19T07:37:41.036 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:37:41.036 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-09-19T07:37:41.036 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-09-19T07:37:41.036 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-09-19T07:37:41.036 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 126, 2024-09-19T07:37:41.036 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 435175, 2024-09-19T07:37:41.037 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-09-19T07:37:41.037 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 126, 2024-09-19T07:37:41.037 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-09-19T07:37:41.037 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-09-19T07:37:41.037 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 126, 2024-09-19T07:37:41.037 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 435175, 2024-09-19T07:37:41.037 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-09-19T07:37:41.037 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-09-19T07:37:41.037 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-09-19T07:37:41.037 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-09-19T07:37:41.037 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:41.037 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-09-19T07:37:41.038 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-09-19T07:37:41.732 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-19T07:37:41.732 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-19T07:37:41.733 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-19T07:37:41.733 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-19T07:37:41.733 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-19T07:37:41.743 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-19T07:37:41.743 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:37:41.743 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:37:41.785 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-19T07:37:41.786 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:37:41.865 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T07:37:42.281 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:42.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:42 smithi203 ceph-mon[35335]: pgmap v515: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T07:37:42.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:42 smithi189 ceph-mon[34851]: pgmap v515: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T07:37:42.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:42 smithi159 ceph-mon[30472]: pgmap v515: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T07:37:42.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.844+0000 7f0e81400640 1 -- 172.21.15.159:0/2923713887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e7c1079c0 msgr2=0x7f0e7c107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:42.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.844+0000 7f0e81400640 1 --2- 172.21.15.159:0/2923713887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e7c1079c0 0x7f0e7c107da0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009930 tx=0x7f0e6c02a420 comp rx=0 tx=0).stop 2024-09-19T07:37:42.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.844+0000 7f0e81400640 1 -- 172.21.15.159:0/2923713887 shutdown_connections 2024-09-19T07:37:42.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.844+0000 7f0e81400640 1 --2- 172.21.15.159:0/2923713887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c102360 0x7f0e7c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:42.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.844+0000 7f0e81400640 1 --2- 172.21.15.159:0/2923713887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e7c1019c0 0x7f0e7c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:42.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.844+0000 7f0e81400640 1 --2- 172.21.15.159:0/2923713887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e7c1079c0 0x7f0e7c107da0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:42.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.844+0000 7f0e81400640 1 -- 172.21.15.159:0/2923713887 >> 172.21.15.159:0/2923713887 conn(0x7f0e7c0fd6e0 msgr2=0x7f0e7c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:42.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.844+0000 7f0e81400640 1 -- 172.21.15.159:0/2923713887 shutdown_connections 2024-09-19T07:37:42.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.845+0000 7f0e81400640 1 -- 172.21.15.159:0/2923713887 wait complete. 2024-09-19T07:37:42.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.845+0000 7f0e81400640 1 Processor -- start 2024-09-19T07:37:42.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.846+0000 7f0e81400640 1 -- start start 2024-09-19T07:37:42.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.846+0000 7f0e81400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e7c1019c0 0x7f0e7c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:42.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.846+0000 7f0e81400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e7c102360 0x7f0e7c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:42.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.846+0000 7f0e81400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c1079c0 0x7f0e7c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:42.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.846+0000 7f0e81400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c111de0 con 0x7f0e7c1019c0 2024-09-19T07:37:42.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.846+0000 7f0e81400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c111c60 con 0x7f0e7c102360 2024-09-19T07:37:42.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.846+0000 7f0e81400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c111f60 con 0x7f0e7c1079c0 2024-09-19T07:37:42.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.846+0000 7f0e7b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e7c102360 0x7f0e7c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:42.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.846+0000 7f0e7be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e7c1019c0 0x7f0e7c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:42.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.846+0000 7f0e7b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e7c102360 0x7f0e7c19a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:34624/0 (socket says 172.21.15.159:34624) 2024-09-19T07:37:42.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.846+0000 7f0e7b400640 1 -- 172.21.15.159:0/651741358 learned_addr learned my addr 172.21.15.159:0/651741358 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:37:42.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.846+0000 7f0e80a00640 1 --2- 172.21.15.159:0/651741358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c1079c0 0x7f0e7c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.847+0000 7f0e7b400640 1 -- 172.21.15.159:0/651741358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c1079c0 msgr2=0x7f0e7c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.847+0000 7f0e7b400640 1 --2- 172.21.15.159:0/651741358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c1079c0 0x7f0e7c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.847+0000 7f0e7b400640 1 -- 172.21.15.159:0/651741358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e7c1019c0 msgr2=0x7f0e7c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.847+0000 7f0e7b400640 1 --2- 172.21.15.159:0/651741358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e7c1019c0 0x7f0e7c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.847+0000 7f0e7b400640 1 -- 172.21.15.159:0/651741358 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7c19f410 con 0x7f0e7c102360 2024-09-19T07:37:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.847+0000 7f0e80a00640 1 --2- 172.21.15.159:0/651741358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c1079c0 0x7f0e7c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.847+0000 7f0e7be00640 1 --2- 172.21.15.159:0/651741358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e7c1019c0 0x7f0e7c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:37:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.847+0000 7f0e7b400640 1 --2- 172.21.15.159:0/651741358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e7c102360 0x7f0e7c19a9a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800e9f0 tx=0x7f0e6800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.847+0000 7f0e78c00640 1 -- 172.21.15.159:0/651741358 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6800cd80 con 0x7f0e7c102360 2024-09-19T07:37:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.847+0000 7f0e81400640 1 -- 172.21.15.159:0/651741358 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e7c19f700 con 0x7f0e7c102360 2024-09-19T07:37:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.847+0000 7f0e78c00640 1 -- 172.21.15.159:0/651741358 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e68010910 con 0x7f0e7c102360 2024-09-19T07:37:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.847+0000 7f0e78c00640 1 -- 172.21.15.159:0/651741358 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e680188f0 con 0x7f0e7c102360 2024-09-19T07:37:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.848+0000 7f0e81400640 1 -- 172.21.15.159:0/651741358 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e7c1a6fe0 con 0x7f0e7c102360 2024-09-19T07:37:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.849+0000 7f0e81400640 1 -- 172.21.15.159:0/651741358 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e7c101e20 con 0x7f0e7c102360 2024-09-19T07:37:42.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.851+0000 7f0e78c00640 1 -- 172.21.15.159:0/651741358 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f0e68010ac0 con 0x7f0e7c102360 2024-09-19T07:37:42.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.853+0000 7f0e78c00640 1 --2- 172.21.15.159:0/651741358 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0e600a32c0 0x7f0e600a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:42.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.853+0000 7f0e78c00640 1 -- 172.21.15.159:0/651741358 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f0e680d03e0 con 0x7f0e7c102360 2024-09-19T07:37:42.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.853+0000 7f0e7be00640 1 --2- 172.21.15.159:0/651741358 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0e600a32c0 0x7f0e600a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:42.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.853+0000 7f0e7be00640 1 --2- 172.21.15.159:0/651741358 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0e600a32c0 0x7f0e600a5780 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009900 tx=0x7f0e6c0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:42.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:42.854+0000 7f0e78c00640 1 -- 172.21.15.159:0/651741358 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0e6809ccf0 con 0x7f0e7c102360 2024-09-19T07:37:43.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:43.041+0000 7f0e81400640 1 -- 172.21.15.159:0/651741358 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0e7c1a7280 con 0x7f0e7c102360 2024-09-19T07:37:43.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:43.042+0000 7f0e78c00640 1 -- 172.21.15.159:0/651741358 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v100) ==== 81+0+600 (secure 0 0 0) 0x7f0e6809c5f0 con 0x7f0e7c102360 2024-09-19T07:37:43.043 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/2383222735 2024-09-20T07:23:30.596931+0000 2024-09-19T07:37:43.043 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/1561371050 2024-09-20T07:23:30.596931+0000 2024-09-19T07:37:43.044 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6801/2966947684 2024-09-20T07:23:30.596931+0000 2024-09-19T07:37:43.044 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6800/2966947684 2024-09-20T07:23:30.596931+0000 2024-09-19T07:37:43.044 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6801/432893324 2024-09-20T07:23:03.089190+0000 2024-09-19T07:37:43.044 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6800/432893324 2024-09-20T07:23:03.089190+0000 2024-09-19T07:37:43.044 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/2108730247 2024-09-20T07:23:03.089190+0000 2024-09-19T07:37:43.044 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/3393992615 2024-09-20T07:23:30.596931+0000 2024-09-19T07:37:43.044 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/1366863724 2024-09-20T07:23:03.089190+0000 2024-09-19T07:37:43.044 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:0/1628344860 2024-09-20T07:23:03.089190+0000 2024-09-19T07:37:43.044 INFO:teuthology.orchestra.run.smithi159.stderr:listed 10 entries 2024-09-19T07:37:43.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:43.046+0000 7f0e81400640 1 -- 172.21.15.159:0/651741358 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0e600a32c0 msgr2=0x7f0e600a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:43.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:43.046+0000 7f0e81400640 1 --2- 172.21.15.159:0/651741358 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0e600a32c0 0x7f0e600a5780 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009900 tx=0x7f0e6c0058d0 comp rx=0 tx=0).stop 2024-09-19T07:37:43.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:43.046+0000 7f0e81400640 1 -- 172.21.15.159:0/651741358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e7c102360 msgr2=0x7f0e7c19a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:43.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:43.046+0000 7f0e81400640 1 --2- 172.21.15.159:0/651741358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e7c102360 0x7f0e7c19a9a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800e9f0 tx=0x7f0e6800eeb0 comp rx=0 tx=0).stop 2024-09-19T07:37:43.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:43.046+0000 7f0e81400640 1 -- 172.21.15.159:0/651741358 shutdown_connections 2024-09-19T07:37:43.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:43.046+0000 7f0e81400640 1 --2- 172.21.15.159:0/651741358 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f0e600a32c0 0x7f0e600a5780 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:43.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:43.046+0000 7f0e81400640 1 --2- 172.21.15.159:0/651741358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c1079c0 0x7f0e7c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:43.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:43.046+0000 7f0e81400640 1 --2- 172.21.15.159:0/651741358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e7c102360 0x7f0e7c19a9a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:43.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:43.046+0000 7f0e81400640 1 --2- 172.21.15.159:0/651741358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e7c1019c0 0x7f0e7c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:43.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:43.046+0000 7f0e81400640 1 -- 172.21.15.159:0/651741358 >> 172.21.15.159:0/651741358 conn(0x7f0e7c0fd6e0 msgr2=0x7f0e7c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:43.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:43.046+0000 7f0e81400640 1 -- 172.21.15.159:0/651741358 shutdown_connections 2024-09-19T07:37:43.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:43.046+0000 7f0e81400640 1 -- 172.21.15.159:0/651741358 wait complete. 2024-09-19T07:37:44.044 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-19T07:37:44.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:37:44.044 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-19T07:37:44.130 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-09-19T07:37:44.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:37:44.130 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T07:37:44.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:37:44.179 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-09-19T07:37:44.215 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T07:37:44.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:37:44.215 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T07:37:44.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:44 smithi203 ceph-mon[35335]: pgmap v516: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:37:44.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:44 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/651741358' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T07:37:44.303 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:44 smithi159 ceph-mon[30472]: pgmap v516: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:37:44.303 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:44 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/651741358' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T07:37:44.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:44 smithi189 ceph-mon[34851]: pgmap v516: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-19T07:37:44.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:44 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/651741358' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T07:37:44.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:37:44.313 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link delete ceph-brx 2024-09-19T07:37:44.638 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T07:37:44.647 INFO:tasks.cephadm:Teardown begin 2024-09-19T07:37:44.648 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T07:37:44.648 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:37:45.026 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:46.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:46 smithi203 ceph-mon[35335]: pgmap v517: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:37:46.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:46 smithi189 ceph-mon[34851]: pgmap v517: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:37:46.367 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:46 smithi159 ceph-mon[30472]: pgmap v517: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T07:37:46.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.482+0000 7f2ebee00640 1 -- 172.21.15.159:0/2933530606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eb8109120 msgr2=0x7f2eb8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:46.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.482+0000 7f2ebee00640 1 --2- 172.21.15.159:0/2933530606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eb8109120 0x7f2eb8110ca0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009ca0 tx=0x7f2eb002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.482+0000 7f2ebee00640 1 -- 172.21.15.159:0/2933530606 shutdown_connections 2024-09-19T07:37:46.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.482+0000 7f2ebee00640 1 --2- 172.21.15.159:0/2933530606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eb8109120 0x7f2eb8110ca0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.482+0000 7f2ebee00640 1 --2- 172.21.15.159:0/2933530606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eb81049a0 0x7f2eb81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.482+0000 7f2ebee00640 1 --2- 172.21.15.159:0/2933530606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eb8103ff0 0x7f2eb81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.482+0000 7f2ebee00640 1 -- 172.21.15.159:0/2933530606 >> 172.21.15.159:0/2933530606 conn(0x7f2eb80ffa90 msgr2=0x7f2eb8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:46.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.482+0000 7f2ebee00640 1 -- 172.21.15.159:0/2933530606 shutdown_connections 2024-09-19T07:37:46.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.482+0000 7f2ebee00640 1 -- 172.21.15.159:0/2933530606 wait complete. 2024-09-19T07:37:46.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.483+0000 7f2ebee00640 1 Processor -- start 2024-09-19T07:37:46.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.483+0000 7f2ebee00640 1 -- start start 2024-09-19T07:37:46.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.484+0000 7f2ebee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eb8103ff0 0x7f2eb819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.484+0000 7f2ebee00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eb81049a0 0x7f2eb819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.484+0000 7f2ebee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eb8109120 0x7f2eb819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.484+0000 7f2ebee00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb81141a0 con 0x7f2eb8109120 2024-09-19T07:37:46.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.484+0000 7f2ebee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb8114020 con 0x7f2eb8103ff0 2024-09-19T07:37:46.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.484+0000 7f2ebee00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb8114320 con 0x7f2eb81049a0 2024-09-19T07:37:46.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.484+0000 7f2eb7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eb8103ff0 0x7f2eb819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.484+0000 7f2eb7400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eb81049a0 0x7f2eb819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.484+0000 7f2eb7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eb8103ff0 0x7f2eb819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.159:34640/0 (socket says 172.21.15.159:34640) 2024-09-19T07:37:46.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.484+0000 7f2ebce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eb8109120 0x7f2eb819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.484+0000 7f2eb7e00640 1 -- 172.21.15.159:0/963801633 learned_addr learned my addr 172.21.15.159:0/963801633 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:37:46.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.484+0000 7f2eb7e00640 1 -- 172.21.15.159:0/963801633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eb81049a0 msgr2=0x7f2eb819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:46.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.484+0000 7f2eb7e00640 1 --2- 172.21.15.159:0/963801633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eb81049a0 0x7f2eb819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.484+0000 7f2eb7e00640 1 -- 172.21.15.159:0/963801633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eb8109120 msgr2=0x7f2eb819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:46.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.485+0000 7f2eb7e00640 1 --2- 172.21.15.159:0/963801633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eb8109120 0x7f2eb819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.485+0000 7f2eb7e00640 1 -- 172.21.15.159:0/963801633 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb819f570 con 0x7f2eb8103ff0 2024-09-19T07:37:46.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.485+0000 7f2eb7400640 1 --2- 172.21.15.159:0/963801633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eb81049a0 0x7f2eb819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:37:46.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.485+0000 7f2eb7e00640 1 --2- 172.21.15.159:0/963801633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eb8103ff0 0x7f2eb819a5c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800e9f0 tx=0x7f2ea800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:46.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.485+0000 7f2eb4c00640 1 -- 172.21.15.159:0/963801633 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800cd80 con 0x7f2eb8103ff0 2024-09-19T07:37:46.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.485+0000 7f2eb4c00640 1 -- 172.21.15.159:0/963801633 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ea8005500 con 0x7f2eb8103ff0 2024-09-19T07:37:46.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.485+0000 7f2ebee00640 1 -- 172.21.15.159:0/963801633 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb819f860 con 0x7f2eb8103ff0 2024-09-19T07:37:46.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.485+0000 7f2eb4c00640 1 -- 172.21.15.159:0/963801633 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8010690 con 0x7f2eb8103ff0 2024-09-19T07:37:46.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.485+0000 7f2ebee00640 1 -- 172.21.15.159:0/963801633 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb81a7140 con 0x7f2eb8103ff0 2024-09-19T07:37:46.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.486+0000 7f2ebee00640 1 -- 172.21.15.159:0/963801633 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb81043d0 con 0x7f2eb8103ff0 2024-09-19T07:37:46.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.488+0000 7f2eb4c00640 1 -- 172.21.15.159:0/963801633 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f2ea80046c0 con 0x7f2eb8103ff0 2024-09-19T07:37:46.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.490+0000 7f2eb4c00640 1 --2- 172.21.15.159:0/963801633 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2e9c0a3240 0x7f2e9c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.490+0000 7f2eb4c00640 1 -- 172.21.15.159:0/963801633 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f2ea80d03f0 con 0x7f2eb8103ff0 2024-09-19T07:37:46.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.490+0000 7f2eb7400640 1 --2- 172.21.15.159:0/963801633 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2e9c0a3240 0x7f2e9c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.491+0000 7f2eb7400640 1 --2- 172.21.15.159:0/963801633 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2e9c0a3240 0x7f2e9c0a5700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2eb810aaf0 tx=0x7f2ea400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:46.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.492+0000 7f2eb4c00640 1 -- 172.21.15.159:0/963801633 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2ea809cd00 con 0x7f2eb8103ff0 2024-09-19T07:37:46.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.702+0000 7f2ebee00640 1 -- 172.21.15.159:0/963801633 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2eb819b920 con 0x7f2eb8103ff0 2024-09-19T07:37:46.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.704+0000 7f2eb4c00640 1 -- 172.21.15.159:0/963801633 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11936 (secure 0 0 0) 0x7f2ea809c600 con 0x7f2eb8103ff0 2024-09-19T07:37:46.705 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:37:46.705 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":26,"btime":"2024-09-19T07:33:50:885906+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3464449771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3464449771},{"type":"v1","addr":"172.21.15.159:6837","nonce":3464449771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2036264856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2036264856},{"type":"v1","addr":"172.21.15.159:6839","nonce":2036264856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6841/2874566405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6840","nonce":2874566405},{"type":"v1","addr":"172.21.15.159:6841","nonce":2874566405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:31:03.518840+0000","modified":"2024-09-19T07:33:50.168288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24542,"mds_1":24514,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/1340279549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1340279549},{"type":"v1","addr":"172.21.15.189:6835","nonce":1340279549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/937970878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":937970878},{"type":"v1","addr":"172.21.15.203:6835","nonce":937970878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1053443810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1053443810},{"type":"v1","addr":"172.21.15.203:6837","nonce":1053443810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.189:6837/1030825585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1030825585},{"type":"v1","addr":"172.21.15.189:6837","nonce":1030825585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.203:6839/3862781462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":3862781462},{"type":"v1","addr":"172.21.15.203:6839","nonce":3862781462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.189:6839/2361818663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2361818663},{"type":"v1","addr":"172.21.15.189:6839","nonce":2361818663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24514,24506]},"id":1}]} 2024-09-19T07:37:46.705 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 26 2024-09-19T07:37:46.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.706+0000 7f2ebee00640 1 -- 172.21.15.159:0/963801633 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2e9c0a3240 msgr2=0x7f2e9c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:46.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.706+0000 7f2ebee00640 1 --2- 172.21.15.159:0/963801633 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2e9c0a3240 0x7f2e9c0a5700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2eb810aaf0 tx=0x7f2ea400b410 comp rx=0 tx=0).stop 2024-09-19T07:37:46.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.706+0000 7f2ebee00640 1 -- 172.21.15.159:0/963801633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eb8103ff0 msgr2=0x7f2eb819a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:46.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.706+0000 7f2ebee00640 1 --2- 172.21.15.159:0/963801633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eb8103ff0 0x7f2eb819a5c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800e9f0 tx=0x7f2ea800edf0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.706+0000 7f2ebee00640 1 -- 172.21.15.159:0/963801633 shutdown_connections 2024-09-19T07:37:46.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.706+0000 7f2ebee00640 1 --2- 172.21.15.159:0/963801633 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f2e9c0a3240 0x7f2e9c0a5700 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.706+0000 7f2ebee00640 1 --2- 172.21.15.159:0/963801633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eb8109120 0x7f2eb819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.706+0000 7f2ebee00640 1 --2- 172.21.15.159:0/963801633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2eb81049a0 0x7f2eb819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.706+0000 7f2ebee00640 1 --2- 172.21.15.159:0/963801633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eb8103ff0 0x7f2eb819a5c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.706+0000 7f2ebee00640 1 -- 172.21.15.159:0/963801633 >> 172.21.15.159:0/963801633 conn(0x7f2eb80ffa90 msgr2=0x7f2eb81005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:46.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.707+0000 7f2ebee00640 1 -- 172.21.15.159:0/963801633 shutdown_connections 2024-09-19T07:37:46.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:46.707+0000 7f2ebee00640 1 -- 172.21.15.159:0/963801633 wait complete. 2024-09-19T07:37:47.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:47 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/963801633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:47.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:47 smithi203 ceph-mon[35335]: pgmap v518: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-19T07:37:47.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:47 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/963801633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:47.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:47 smithi189 ceph-mon[34851]: pgmap v518: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-19T07:37:47.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:47 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/963801633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:47.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:47 smithi159 ceph-mon[30472]: pgmap v518: 129 pgs: 129 active+clean; 1.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-19T07:37:47.593 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:37:47.968 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:48.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.562+0000 7f711ca00640 1 -- 172.21.15.159:0/4193238047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71181019c0 msgr2=0x7f7118101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:48.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.562+0000 7f711ca00640 1 --2- 172.21.15.159:0/4193238047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71181019c0 0x7f7118101e20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7100009890 tx=0x7f710002a360 comp rx=0 tx=0).stop 2024-09-19T07:37:48.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.562+0000 7f711ca00640 1 -- 172.21.15.159:0/4193238047 shutdown_connections 2024-09-19T07:37:48.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.562+0000 7f711ca00640 1 --2- 172.21.15.159:0/4193238047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7118102360 0x7f711810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:48.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.562+0000 7f711ca00640 1 --2- 172.21.15.159:0/4193238047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71181019c0 0x7f7118101e20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:48.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.562+0000 7f711ca00640 1 --2- 172.21.15.159:0/4193238047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71181079c0 0x7f7118107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:48.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.562+0000 7f711ca00640 1 -- 172.21.15.159:0/4193238047 >> 172.21.15.159:0/4193238047 conn(0x7f71180fd6e0 msgr2=0x7f71180ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:48.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.562+0000 7f711ca00640 1 -- 172.21.15.159:0/4193238047 shutdown_connections 2024-09-19T07:37:48.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.562+0000 7f711ca00640 1 -- 172.21.15.159:0/4193238047 wait complete. 2024-09-19T07:37:48.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.563+0000 7f711ca00640 1 Processor -- start 2024-09-19T07:37:48.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.564+0000 7f711ca00640 1 -- start start 2024-09-19T07:37:48.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.564+0000 7f711ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71181019c0 0x7f71180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:48.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.564+0000 7f711ca00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7118102360 0x7f7118074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:48.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.564+0000 7f711ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71181079c0 0x7f7118074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:48.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.564+0000 7f711ca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7118111dc0 con 0x7f71181079c0 2024-09-19T07:37:48.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.564+0000 7f711ca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7118111c40 con 0x7f71181019c0 2024-09-19T07:37:48.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.564+0000 7f711ca00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7118111f40 con 0x7f7118102360 2024-09-19T07:37:48.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.564+0000 7f710de00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7118102360 0x7f7118074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:48.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.564+0000 7f7116800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71181019c0 0x7f71180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:48.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.564+0000 7f710de00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7118102360 0x7f7118074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:34566/0 (socket says 172.21.15.159:34566) 2024-09-19T07:37:48.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.564+0000 7f710de00640 1 -- 172.21.15.159:0/595048218 learned_addr learned my addr 172.21.15.159:0/595048218 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:37:48.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.564+0000 7f7117200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71181079c0 0x7f7118074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:48.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.565+0000 7f710de00640 1 -- 172.21.15.159:0/595048218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71181019c0 msgr2=0x7f71180744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:48.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.565+0000 7f710de00640 1 --2- 172.21.15.159:0/595048218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71181019c0 0x7f71180744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:48.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.565+0000 7f710de00640 1 -- 172.21.15.159:0/595048218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71181079c0 msgr2=0x7f7118074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:48.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.565+0000 7f710de00640 1 --2- 172.21.15.159:0/595048218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71181079c0 0x7f7118074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:48.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.565+0000 7f710de00640 1 -- 172.21.15.159:0/595048218 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71181a67a0 con 0x7f7118102360 2024-09-19T07:37:48.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.565+0000 7f7116800640 1 --2- 172.21.15.159:0/595048218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71181019c0 0x7f71180744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:37:48.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.565+0000 7f710de00640 1 --2- 172.21.15.159:0/595048218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7118102360 0x7f7118074a30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7100035040 tx=0x7f7100033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:48.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.565+0000 7f710fe00640 1 -- 172.21.15.159:0/595048218 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100037040 con 0x7f7118102360 2024-09-19T07:37:48.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.565+0000 7f711ca00640 1 -- 172.21.15.159:0/595048218 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71181a6a30 con 0x7f7118102360 2024-09-19T07:37:48.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.565+0000 7f710fe00640 1 -- 172.21.15.159:0/595048218 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7100040070 con 0x7f7118102360 2024-09-19T07:37:48.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.565+0000 7f710fe00640 1 -- 172.21.15.159:0/595048218 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71000039b0 con 0x7f7118102360 2024-09-19T07:37:48.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.566+0000 7f711ca00640 1 -- 172.21.15.159:0/595048218 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71181a6f10 con 0x7f7118102360 2024-09-19T07:37:48.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.567+0000 7f711ca00640 1 -- 172.21.15.159:0/595048218 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7118101e20 con 0x7f7118102360 2024-09-19T07:37:48.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.569+0000 7f710fe00640 1 -- 172.21.15.159:0/595048218 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f7100030610 con 0x7f7118102360 2024-09-19T07:37:48.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.570+0000 7f710fe00640 1 --2- 172.21.15.159:0/595048218 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f70fc0a3240 0x7f70fc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:48.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.570+0000 7f710fe00640 1 -- 172.21.15.159:0/595048218 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f71000f0200 con 0x7f7118102360 2024-09-19T07:37:48.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.570+0000 7f7116800640 1 --2- 172.21.15.159:0/595048218 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f70fc0a3240 0x7f70fc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:48.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.571+0000 7f7116800640 1 --2- 172.21.15.159:0/595048218 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f70fc0a3240 0x7f70fc0a5700 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f71080096c0 tx=0x7f7108009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:48.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.572+0000 7f710fe00640 1 -- 172.21.15.159:0/595048218 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f71000bcb70 con 0x7f7118102360 2024-09-19T07:37:48.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.782+0000 7f711ca00640 1 -- 172.21.15.159:0/595048218 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7118109f30 con 0x7f7118102360 2024-09-19T07:37:48.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.783+0000 7f710fe00640 1 -- 172.21.15.159:0/595048218 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11936 (secure 0 0 0) 0x7f71000bc470 con 0x7f7118102360 2024-09-19T07:37:48.783 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:37:48.783 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":26,"btime":"2024-09-19T07:33:50:885906+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/205226977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":205226977},{"type":"v1","addr":"172.21.15.159:6835","nonce":205226977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/3464449771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":3464449771},{"type":"v1","addr":"172.21.15.159:6837","nonce":3464449771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14487,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2036264856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2036264856},{"type":"v1","addr":"172.21.15.159:6839","nonce":2036264856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14499,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6841/2874566405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6840","nonce":2874566405},{"type":"v1","addr":"172.21.15.159:6841","nonce":2874566405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1415737112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1415737112},{"type":"v1","addr":"172.21.15.203:6833","nonce":1415737112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/2799019079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":2799019079},{"type":"v1","addr":"172.21.15.189:6833","nonce":2799019079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:31:03.518840+0000","modified":"2024-09-19T07:33:50.168288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24542,"mds_1":24514,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/1340279549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1340279549},{"type":"v1","addr":"172.21.15.189:6835","nonce":1340279549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/937970878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":937970878},{"type":"v1","addr":"172.21.15.203:6835","nonce":937970878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/1053443810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":1053443810},{"type":"v1","addr":"172.21.15.203:6837","nonce":1053443810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":20,"addr":"172.21.15.189:6837/1030825585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1030825585},{"type":"v1","addr":"172.21.15.189:6837","nonce":1030825585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":19,"addr":"172.21.15.203:6839/3862781462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":3862781462},{"type":"v1","addr":"172.21.15.203:6839","nonce":3862781462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.189:6839/2361818663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2361818663},{"type":"v1","addr":"172.21.15.189:6839","nonce":2361818663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24514,24506]},"id":1}]} 2024-09-19T07:37:48.783 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 26 2024-09-19T07:37:48.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.785+0000 7f711ca00640 1 -- 172.21.15.159:0/595048218 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f70fc0a3240 msgr2=0x7f70fc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:48.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.785+0000 7f711ca00640 1 --2- 172.21.15.159:0/595048218 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f70fc0a3240 0x7f70fc0a5700 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f71080096c0 tx=0x7f7108009260 comp rx=0 tx=0).stop 2024-09-19T07:37:48.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.785+0000 7f711ca00640 1 -- 172.21.15.159:0/595048218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7118102360 msgr2=0x7f7118074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:48.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.785+0000 7f711ca00640 1 --2- 172.21.15.159:0/595048218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7118102360 0x7f7118074a30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7100035040 tx=0x7f7100033b60 comp rx=0 tx=0).stop 2024-09-19T07:37:48.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.785+0000 7f711ca00640 1 -- 172.21.15.159:0/595048218 shutdown_connections 2024-09-19T07:37:48.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.785+0000 7f711ca00640 1 --2- 172.21.15.159:0/595048218 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f70fc0a3240 0x7f70fc0a5700 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:48.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.785+0000 7f711ca00640 1 --2- 172.21.15.159:0/595048218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71181079c0 0x7f7118074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:48.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.785+0000 7f711ca00640 1 --2- 172.21.15.159:0/595048218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7118102360 0x7f7118074a30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:48.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.785+0000 7f711ca00640 1 --2- 172.21.15.159:0/595048218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71181019c0 0x7f71180744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:48.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.785+0000 7f711ca00640 1 -- 172.21.15.159:0/595048218 >> 172.21.15.159:0/595048218 conn(0x7f71180fd6e0 msgr2=0x7f71180fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:48.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.785+0000 7f711ca00640 1 -- 172.21.15.159:0/595048218 shutdown_connections 2024-09-19T07:37:48.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:48.786+0000 7f711ca00640 1 -- 172.21.15.159:0/595048218 wait complete. 2024-09-19T07:37:49.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:48 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/595048218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:49.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:48 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/595048218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:49.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:48 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/595048218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:49.883 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:37:50.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:49 smithi159 ceph-mon[30472]: pgmap v519: 129 pgs: 129 active+clean; 1.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T07:37:50.187 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:49 smithi203 ceph-mon[35335]: pgmap v519: 129 pgs: 129 active+clean; 1.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T07:37:50.251 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:50.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:49 smithi189 ceph-mon[34851]: pgmap v519: 129 pgs: 129 active+clean; 1.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T07:37:50.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.829+0000 7fcd62e00640 1 -- 172.21.15.159:0/1762771426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd5c104010 msgr2=0x7fcd5c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:50.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.829+0000 7fcd62e00640 1 --2- 172.21.15.159:0/1762771426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd5c104010 0x7fcd5c1043f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009ca0 tx=0x7fcd4c02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.829+0000 7fcd62e00640 1 -- 172.21.15.159:0/1762771426 shutdown_connections 2024-09-19T07:37:50.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.829+0000 7fcd62e00640 1 --2- 172.21.15.159:0/1762771426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c109140 0x7fcd5c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.829+0000 7fcd62e00640 1 --2- 172.21.15.159:0/1762771426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd5c1049c0 0x7fcd5c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.829+0000 7fcd62e00640 1 --2- 172.21.15.159:0/1762771426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd5c104010 0x7fcd5c1043f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.829+0000 7fcd62e00640 1 -- 172.21.15.159:0/1762771426 >> 172.21.15.159:0/1762771426 conn(0x7fcd5c0ffa50 msgr2=0x7fcd5c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:50.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.829+0000 7fcd62e00640 1 -- 172.21.15.159:0/1762771426 shutdown_connections 2024-09-19T07:37:50.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.829+0000 7fcd62e00640 1 -- 172.21.15.159:0/1762771426 wait complete. 2024-09-19T07:37:50.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.830+0000 7fcd62e00640 1 Processor -- start 2024-09-19T07:37:50.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.830+0000 7fcd62e00640 1 -- start start 2024-09-19T07:37:50.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd62e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd5c104010 0x7fcd5c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd62e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c1049c0 0x7fcd5c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd62e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd5c109140 0x7fcd5c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd62e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c114190 con 0x7fcd5c104010 2024-09-19T07:37:50.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd62e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c114010 con 0x7fcd5c109140 2024-09-19T07:37:50.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd62e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c114310 con 0x7fcd5c1049c0 2024-09-19T07:37:50.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd5be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd5c104010 0x7fcd5c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd53400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c1049c0 0x7fcd5c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd53400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c1049c0 0x7fcd5c19ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:34584/0 (socket says 172.21.15.159:34584) 2024-09-19T07:37:50.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd53400640 1 -- 172.21.15.159:0/1595311821 learned_addr learned my addr 172.21.15.159:0/1595311821 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:37:50.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd60e00640 1 --2- 172.21.15.159:0/1595311821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd5c109140 0x7fcd5c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd5be00640 1 -- 172.21.15.159:0/1595311821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c1049c0 msgr2=0x7fcd5c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:50.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd5be00640 1 --2- 172.21.15.159:0/1595311821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c1049c0 0x7fcd5c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd5be00640 1 -- 172.21.15.159:0/1595311821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd5c109140 msgr2=0x7fcd5c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:50.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd5be00640 1 --2- 172.21.15.159:0/1595311821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd5c109140 0x7fcd5c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd5be00640 1 -- 172.21.15.159:0/1595311821 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c19f600 con 0x7fcd5c104010 2024-09-19T07:37:50.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd53400640 1 --2- 172.21.15.159:0/1595311821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c1049c0 0x7fcd5c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:37:50.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.831+0000 7fcd60e00640 1 --2- 172.21.15.159:0/1595311821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd5c109140 0x7fcd5c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:50.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.832+0000 7fcd5be00640 1 --2- 172.21.15.159:0/1595311821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd5c104010 0x7fcd5c19a650 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c000f80 tx=0x7fcd4c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:50.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.832+0000 7fcd59600640 1 -- 172.21.15.159:0/1595311821 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c031650 con 0x7fcd5c104010 2024-09-19T07:37:50.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.832+0000 7fcd59600640 1 -- 172.21.15.159:0/1595311821 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd4c0317f0 con 0x7fcd5c104010 2024-09-19T07:37:50.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.832+0000 7fcd62e00640 1 -- 172.21.15.159:0/1595311821 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5c19f890 con 0x7fcd5c104010 2024-09-19T07:37:50.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.832+0000 7fcd62e00640 1 -- 172.21.15.159:0/1595311821 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5c1a71d0 con 0x7fcd5c104010 2024-09-19T07:37:50.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.832+0000 7fcd59600640 1 -- 172.21.15.159:0/1595311821 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c02fa70 con 0x7fcd5c104010 2024-09-19T07:37:50.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.833+0000 7fcd62e00640 1 -- 172.21.15.159:0/1595311821 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd5c1043f0 con 0x7fcd5c104010 2024-09-19T07:37:50.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.835+0000 7fcd59600640 1 -- 172.21.15.159:0/1595311821 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fcd4c0045b0 con 0x7fcd5c104010 2024-09-19T07:37:50.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.836+0000 7fcd59600640 1 --2- 172.21.15.159:0/1595311821 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcd400a3240 0x7fcd400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.837+0000 7fcd59600640 1 -- 172.21.15.159:0/1595311821 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7fcd4c0f0fd0 con 0x7fcd5c104010 2024-09-19T07:37:50.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.837+0000 7fcd53400640 1 --2- 172.21.15.159:0/1595311821 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcd400a3240 0x7fcd400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.837+0000 7fcd53400640 1 --2- 172.21.15.159:0/1595311821 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcd400a3240 0x7fcd400a5700 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0ffe70 tx=0x7fcd44005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:50.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:50.840+0000 7fcd59600640 1 -- 172.21.15.159:0/1595311821 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcd4c0319a0 con 0x7fcd5c104010 2024-09-19T07:37:51.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:51.019+0000 7fcd62e00640 1 -- 172.21.15.159:0/1595311821 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd5c10c320 con 0x7fcd5c104010 2024-09-19T07:37:51.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:51.020+0000 7fcd59600640 1 -- 172.21.15.159:0/1595311821 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24963 (secure 0 0 0) 0x7fcd4c0bda00 con 0x7fcd5c104010 2024-09-19T07:37:51.021 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-19T07:37:51.021 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":100,"fsid":"c9d18048-7657-11ef-bceb-c7b262605968","created":"2024-09-19T07:22:32.059836+0000","modified":"2024-09-19T07:32:31.832496+0000","last_up_change":"2024-09-19T07:30:03.328697+0000","last_in_change":"2024-09-19T07:29:38.987579+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:25:55.202457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:30:58.022056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:31:01.042290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T07:31:10.977289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"752f5466-fe04-408a-8566-86625947fd3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6803","nonce":2738900830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6805","nonce":2738900830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6809","nonce":2738900830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2738900830},{"type":"v1","addr":"172.21.15.159:6807","nonce":2738900830}]},"public_addr":"172.21.15.159:6803/2738900830","cluster_addr":"172.21.15.159:6805/2738900830","heartbeat_back_addr":"172.21.15.159:6809/2738900830","heartbeat_front_addr":"172.21.15.159:6807/2738900830","state":["exists","up"]},{"osd":1,"uuid":"7513adda-ca82-4df4-aa99-3b050b410424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6801","nonce":265696178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6803","nonce":265696178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6807","nonce":265696178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":265696178},{"type":"v1","addr":"172.21.15.189:6805","nonce":265696178}]},"public_addr":"172.21.15.189:6801/265696178","cluster_addr":"172.21.15.189:6803/265696178","heartbeat_back_addr":"172.21.15.189:6807/265696178","heartbeat_front_addr":"172.21.15.189:6805/265696178","state":["exists","up"]},{"osd":2,"uuid":"aa03be79-58d8-4eb7-a051-0b46c502b12d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6801","nonce":1826565288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6803","nonce":1826565288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6807","nonce":1826565288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1826565288},{"type":"v1","addr":"172.21.15.203:6805","nonce":1826565288}]},"public_addr":"172.21.15.203:6801/1826565288","cluster_addr":"172.21.15.203:6803/1826565288","heartbeat_back_addr":"172.21.15.203:6807/1826565288","heartbeat_front_addr":"172.21.15.203:6805/1826565288","state":["exists","up"]},{"osd":3,"uuid":"fcd734f1-f40f-4566-82ab-c4607039f23f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6811","nonce":109457380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6813","nonce":109457380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6817","nonce":109457380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":109457380},{"type":"v1","addr":"172.21.15.159:6815","nonce":109457380}]},"public_addr":"172.21.15.159:6811/109457380","cluster_addr":"172.21.15.159:6813/109457380","heartbeat_back_addr":"172.21.15.159:6817/109457380","heartbeat_front_addr":"172.21.15.159:6815/109457380","state":["exists","up"]},{"osd":4,"uuid":"bc344c9e-2267-4058-bbe1-9c270327db8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6809","nonce":1036888180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6811","nonce":1036888180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6815","nonce":1036888180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1036888180},{"type":"v1","addr":"172.21.15.189:6813","nonce":1036888180}]},"public_addr":"172.21.15.189:6809/1036888180","cluster_addr":"172.21.15.189:6811/1036888180","heartbeat_back_addr":"172.21.15.189:6815/1036888180","heartbeat_front_addr":"172.21.15.189:6813/1036888180","state":["exists","up"]},{"osd":5,"uuid":"86c48303-929d-42f4-bf48-cbb32649f95f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6809","nonce":4212601512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6811","nonce":4212601512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6815","nonce":4212601512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":4212601512},{"type":"v1","addr":"172.21.15.203:6813","nonce":4212601512}]},"public_addr":"172.21.15.203:6809/4212601512","cluster_addr":"172.21.15.203:6811/4212601512","heartbeat_back_addr":"172.21.15.203:6815/4212601512","heartbeat_front_addr":"172.21.15.203:6813/4212601512","state":["exists","up"]},{"osd":6,"uuid":"f0176ed1-c3a7-4d75-b213-fe88fab4441a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6819","nonce":3817684578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6821","nonce":3817684578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6825","nonce":3817684578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3817684578},{"type":"v1","addr":"172.21.15.159:6823","nonce":3817684578}]},"public_addr":"172.21.15.159:6819/3817684578","cluster_addr":"172.21.15.159:6821/3817684578","heartbeat_back_addr":"172.21.15.159:6825/3817684578","heartbeat_front_addr":"172.21.15.159:6823/3817684578","state":["exists","up"]},{"osd":7,"uuid":"921bd59d-9415-4559-8e9f-035d336afd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6817","nonce":3936826456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6819","nonce":3936826456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6823","nonce":3936826456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3936826456},{"type":"v1","addr":"172.21.15.189:6821","nonce":3936826456}]},"public_addr":"172.21.15.189:6817/3936826456","cluster_addr":"172.21.15.189:6819/3936826456","heartbeat_back_addr":"172.21.15.189:6823/3936826456","heartbeat_front_addr":"172.21.15.189:6821/3936826456","state":["exists","up"]},{"osd":8,"uuid":"ed346bdd-abdd-4f3b-9a19-8ebdc7cfa4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6817","nonce":4062926645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6819","nonce":4062926645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6823","nonce":4062926645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":4062926645},{"type":"v1","addr":"172.21.15.203:6821","nonce":4062926645}]},"public_addr":"172.21.15.203:6817/4062926645","cluster_addr":"172.21.15.203:6819/4062926645","heartbeat_back_addr":"172.21.15.203:6823/4062926645","heartbeat_front_addr":"172.21.15.203:6821/4062926645","state":["exists","up"]},{"osd":9,"uuid":"df29a10f-1c60-41cb-9c2d-bc9f1cb174bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6827","nonce":4185848747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6829","nonce":4185848747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6833","nonce":4185848747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4185848747},{"type":"v1","addr":"172.21.15.159:6831","nonce":4185848747}]},"public_addr":"172.21.15.159:6827/4185848747","cluster_addr":"172.21.15.159:6829/4185848747","heartbeat_back_addr":"172.21.15.159:6833/4185848747","heartbeat_front_addr":"172.21.15.159:6831/4185848747","state":["exists","up"]},{"osd":10,"uuid":"ccc440b4-4682-4998-bb53-95e24423cd1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6825","nonce":3018377605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6827","nonce":3018377605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6831","nonce":3018377605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3018377605},{"type":"v1","addr":"172.21.15.189:6829","nonce":3018377605}]},"public_addr":"172.21.15.189:6825/3018377605","cluster_addr":"172.21.15.189:6827/3018377605","heartbeat_back_addr":"172.21.15.189:6831/3018377605","heartbeat_front_addr":"172.21.15.189:6829/3018377605","state":["exists","up"]},{"osd":11,"uuid":"e84afc5f-8051-4341-8fd0-3afb482ab4df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6825","nonce":2807503891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6827","nonce":2807503891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6831","nonce":2807503891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2807503891},{"type":"v1","addr":"172.21.15.203:6829","nonce":2807503891}]},"public_addr":"172.21.15.203:6825/2807503891","cluster_addr":"172.21.15.203:6827/2807503891","heartbeat_back_addr":"172.21.15.203:6831/2807503891","heartbeat_front_addr":"172.21.15.203:6829/2807503891","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:00.093677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:25.477205+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:25:51.345020+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:21.077603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:26:48.822640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:13.516029+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:27:44.382228+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:10.810700+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:28:36.817155+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:07.728553+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:29:34.747540+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:01.249332+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:0/2383222735":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1561371050":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6800/2966947684":"2024-09-20T07:23:30.596931+0000","172.21.15.159:6801/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:6800/432893324":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/2108730247":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/3393992615":"2024-09-20T07:23:30.596931+0000","172.21.15.159:0/1366863724":"2024-09-20T07:23:03.089190+0000","172.21.15.159:0/1628344860":"2024-09-20T07:23:03.089190+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:37:51.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:51.023+0000 7fcd62e00640 1 -- 172.21.15.159:0/1595311821 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcd400a3240 msgr2=0x7fcd400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:51.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:51.023+0000 7fcd62e00640 1 --2- 172.21.15.159:0/1595311821 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcd400a3240 0x7fcd400a5700 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0ffe70 tx=0x7fcd44005f90 comp rx=0 tx=0).stop 2024-09-19T07:37:51.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:51.023+0000 7fcd62e00640 1 -- 172.21.15.159:0/1595311821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd5c104010 msgr2=0x7fcd5c19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:51.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:51.023+0000 7fcd62e00640 1 --2- 172.21.15.159:0/1595311821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd5c104010 0x7fcd5c19a650 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c000f80 tx=0x7fcd4c003f10 comp rx=0 tx=0).stop 2024-09-19T07:37:51.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:51.023+0000 7fcd62e00640 1 -- 172.21.15.159:0/1595311821 shutdown_connections 2024-09-19T07:37:51.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:51.023+0000 7fcd62e00640 1 --2- 172.21.15.159:0/1595311821 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fcd400a3240 0x7fcd400a5700 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:51.023+0000 7fcd62e00640 1 --2- 172.21.15.159:0/1595311821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd5c109140 0x7fcd5c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:51.023+0000 7fcd62e00640 1 --2- 172.21.15.159:0/1595311821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c1049c0 0x7fcd5c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:51.023+0000 7fcd62e00640 1 --2- 172.21.15.159:0/1595311821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd5c104010 0x7fcd5c19a650 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:51.023+0000 7fcd62e00640 1 -- 172.21.15.159:0/1595311821 >> 172.21.15.159:0/1595311821 conn(0x7fcd5c0ffa50 msgr2=0x7fcd5c100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:51.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:51.023+0000 7fcd62e00640 1 -- 172.21.15.159:0/1595311821 shutdown_connections 2024-09-19T07:37:51.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:51.023+0000 7fcd62e00640 1 -- 172.21.15.159:0/1595311821 wait complete. 2024-09-19T07:37:51.896 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-19T07:37:52.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:51 smithi159 ceph-mon[30472]: pgmap v520: 129 pgs: 129 active+clean; 1.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T07:37:52.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:51 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1595311821' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:37:52.266 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:52.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:51 smithi203 ceph-mon[35335]: pgmap v520: 129 pgs: 129 active+clean; 1.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T07:37:52.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:51 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1595311821' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:37:52.305 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:51 smithi189 ceph-mon[34851]: pgmap v520: 129 pgs: 129 active+clean; 1.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T07:37:52.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:51 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1595311821' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:37:52.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.841+0000 7f73b0a00640 1 -- 172.21.15.159:0/3818638708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73ac101de0 msgr2=0x7f73ac1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:52.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.841+0000 7f73b0a00640 1 --2- 172.21.15.159:0/3818638708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73ac101de0 0x7f73ac1021c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f739c009900 tx=0x7f739c02a410 comp rx=0 tx=0).stop 2024-09-19T07:37:52.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.841+0000 7f73b0a00640 1 -- 172.21.15.159:0/3818638708 shutdown_connections 2024-09-19T07:37:52.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.841+0000 7f73b0a00640 1 --2- 172.21.15.159:0/3818638708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ac106f10 0x7f73ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:52.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.841+0000 7f73b0a00640 1 --2- 172.21.15.159:0/3818638708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73ac102790 0x7f73ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:52.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.841+0000 7f73b0a00640 1 --2- 172.21.15.159:0/3818638708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73ac101de0 0x7f73ac1021c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:52.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.841+0000 7f73b0a00640 1 -- 172.21.15.159:0/3818638708 >> 172.21.15.159:0/3818638708 conn(0x7f73ac0fd8c0 msgr2=0x7f73ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:52.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.841+0000 7f73b0a00640 1 -- 172.21.15.159:0/3818638708 shutdown_connections 2024-09-19T07:37:52.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.841+0000 7f73b0a00640 1 -- 172.21.15.159:0/3818638708 wait complete. 2024-09-19T07:37:52.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.842+0000 7f73b0a00640 1 Processor -- start 2024-09-19T07:37:52.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.842+0000 7f73b0a00640 1 -- start start 2024-09-19T07:37:52.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.843+0000 7f73b0a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73ac101de0 0x7f73ac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:52.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.843+0000 7f73b0a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73ac102790 0x7f73ac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:52.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.843+0000 7f73b0a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ac106f10 0x7f73ac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:52.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.843+0000 7f73b0a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac111fc0 con 0x7f73ac101de0 2024-09-19T07:37:52.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.843+0000 7f73b0a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac111e40 con 0x7f73ac106f10 2024-09-19T07:37:52.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.843+0000 7f73b0a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac112140 con 0x7f73ac102790 2024-09-19T07:37:52.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.843+0000 7f73a9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73ac101de0 0x7f73ac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:52.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.843+0000 7f73aa800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ac106f10 0x7f73ac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:52.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.843+0000 7f73a1400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73ac102790 0x7f73ac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:52.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.843+0000 7f73a9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73ac101de0 0x7f73ac0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43010/0 (socket says 172.21.15.159:43010) 2024-09-19T07:37:52.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.843+0000 7f73a9e00640 1 -- 172.21.15.159:0/461881330 learned_addr learned my addr 172.21.15.159:0/461881330 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:37:52.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.844+0000 7f73aa800640 1 -- 172.21.15.159:0/461881330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73ac102790 msgr2=0x7f73ac074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:52.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.844+0000 7f73aa800640 1 --2- 172.21.15.159:0/461881330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73ac102790 0x7f73ac074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:52.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.844+0000 7f73aa800640 1 -- 172.21.15.159:0/461881330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73ac101de0 msgr2=0x7f73ac0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:52.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.844+0000 7f73aa800640 1 --2- 172.21.15.159:0/461881330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73ac101de0 0x7f73ac0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:52.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.844+0000 7f73aa800640 1 -- 172.21.15.159:0/461881330 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ac1a6900 con 0x7f73ac106f10 2024-09-19T07:37:52.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.844+0000 7f73aa800640 1 --2- 172.21.15.159:0/461881330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ac106f10 0x7f73ac074f70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009fd0 tx=0x7f73a400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:52.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.844+0000 7f73a3400640 1 -- 172.21.15.159:0/461881330 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a400ce20 con 0x7f73ac106f10 2024-09-19T07:37:52.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.844+0000 7f73b0a00640 1 -- 172.21.15.159:0/461881330 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ac1a6bf0 con 0x7f73ac106f10 2024-09-19T07:37:52.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.844+0000 7f73a3400640 1 -- 172.21.15.159:0/461881330 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a4005020 con 0x7f73ac106f10 2024-09-19T07:37:52.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.844+0000 7f73a3400640 1 -- 172.21.15.159:0/461881330 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4010720 con 0x7f73ac106f10 2024-09-19T07:37:52.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.844+0000 7f73b0a00640 1 -- 172.21.15.159:0/461881330 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ac1a7130 con 0x7f73ac106f10 2024-09-19T07:37:52.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.845+0000 7f73b0a00640 1 -- 172.21.15.159:0/461881330 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ac1021c0 con 0x7f73ac106f10 2024-09-19T07:37:52.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.848+0000 7f73a3400640 1 -- 172.21.15.159:0/461881330 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f73a4005540 con 0x7f73ac106f10 2024-09-19T07:37:52.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.849+0000 7f73a3400640 1 --2- 172.21.15.159:0/461881330 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f73900a3240 0x7f73900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:52.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.849+0000 7f73a3400640 1 -- 172.21.15.159:0/461881330 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9179+0+0 (secure 0 0 0) 0x7f73a40d02e0 con 0x7f73ac106f10 2024-09-19T07:37:52.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.849+0000 7f73a9e00640 1 --2- 172.21.15.159:0/461881330 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f73900a3240 0x7f73900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:52.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.850+0000 7f73a9e00640 1 --2- 172.21.15.159:0/461881330 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f73900a3240 0x7f73900a5700 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f739c0096c0 tx=0x7f739c0046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:52.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:52.851+0000 7f73a3400640 1 -- 172.21.15.159:0/461881330 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f73a409cc50 con 0x7f73ac106f10 2024-09-19T07:37:53.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:53.061+0000 7f73b0a00640 1 -- 172.21.15.159:0/461881330 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f73ac02d7c0 con 0x7f73ac106f10 2024-09-19T07:37:53.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:53.072+0000 7f73a3400640 1 -- 172.21.15.159:0/461881330 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f73a409c550 con 0x7f73ac106f10 2024-09-19T07:37:53.073 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T07:37:53.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:53.075+0000 7f73b0a00640 1 -- 172.21.15.159:0/461881330 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f73900a3240 msgr2=0x7f73900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:53.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:53.075+0000 7f73b0a00640 1 --2- 172.21.15.159:0/461881330 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f73900a3240 0x7f73900a5700 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f739c0096c0 tx=0x7f739c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:37:53.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:53.075+0000 7f73b0a00640 1 -- 172.21.15.159:0/461881330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ac106f10 msgr2=0x7f73ac074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:53.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:53.075+0000 7f73b0a00640 1 --2- 172.21.15.159:0/461881330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ac106f10 0x7f73ac074f70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009fd0 tx=0x7f73a400ef10 comp rx=0 tx=0).stop 2024-09-19T07:37:53.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:53.075+0000 7f73b0a00640 1 -- 172.21.15.159:0/461881330 shutdown_connections 2024-09-19T07:37:53.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:53.075+0000 7f73b0a00640 1 --2- 172.21.15.159:0/461881330 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f73900a3240 0x7f73900a5700 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:53.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:53.075+0000 7f73b0a00640 1 --2- 172.21.15.159:0/461881330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ac106f10 0x7f73ac074f70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:53.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:53.075+0000 7f73b0a00640 1 --2- 172.21.15.159:0/461881330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73ac102790 0x7f73ac074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:53.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:53.075+0000 7f73b0a00640 1 --2- 172.21.15.159:0/461881330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73ac101de0 0x7f73ac0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:53.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:53.075+0000 7f73b0a00640 1 -- 172.21.15.159:0/461881330 >> 172.21.15.159:0/461881330 conn(0x7f73ac0fd8c0 msgr2=0x7f73ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:53.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:53.075+0000 7f73b0a00640 1 -- 172.21.15.159:0/461881330 shutdown_connections 2024-09-19T07:37:53.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:53.075+0000 7f73b0a00640 1 -- 172.21.15.159:0/461881330 wait complete. 2024-09-19T07:37:53.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:53 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-a[30468]: 2024-09-19T07:37:53.063+0000 7f5f9de00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:37:53.536 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-c[65479]: ignoring --setuser ceph since I am not root 2024-09-19T07:37:53.536 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-c[65479]: ignoring --setgroup ceph since I am not root 2024-09-19T07:37:53.536 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-mds[65483]: -- 172.21.15.203:0/1989696944 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560d991c3c20 con 0x560d991a1000 2024-09-19T07:37:53.536 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-c[65479]: starting mds.c at 2024-09-19T07:37:53.537 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-i[65781]: ignoring --setuser ceph since I am not root 2024-09-19T07:37:53.537 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-i[65781]: ignoring --setgroup ceph since I am not root 2024-09-19T07:37:53.537 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-mds[65785]: -- 172.21.15.203:0/665101594 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d1dd499c20 con 0x55d1dd477000 2024-09-19T07:37:53.537 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-i[65781]: starting mds.i at 2024-09-19T07:37:53.537 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-f[65177]: ignoring --setuser ceph since I am not root 2024-09-19T07:37:53.537 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-f[65177]: ignoring --setgroup ceph since I am not root 2024-09-19T07:37:53.537 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-mds[65181]: -- 172.21.15.203:0/2293726386 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55572dd0f860 con 0x55572dced400 2024-09-19T07:37:53.537 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-f[65177]: starting mds.f at 2024-09-19T07:37:53.556 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-h[64391]: ignoring --setuser ceph since I am not root 2024-09-19T07:37:53.556 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-h[64391]: ignoring --setgroup ceph since I am not root 2024-09-19T07:37:53.556 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-mds[64395]: -- 172.21.15.189:0/2104531878 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562ff9d8f860 con 0x562ff9d6d400 2024-09-19T07:37:53.556 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-h[64391]: starting mds.h at 2024-09-19T07:37:53.556 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-b[64691]: ignoring --setuser ceph since I am not root 2024-09-19T07:37:53.556 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-b[64691]: ignoring --setgroup ceph since I am not root 2024-09-19T07:37:53.556 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-mds[64695]: -- 172.21.15.189:0/4045687794 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560c10a57860 con 0x560c10a35000 2024-09-19T07:37:53.556 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-b[64691]: starting mds.b at 2024-09-19T07:37:53.557 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-e[64090]: ignoring --setuser ceph since I am not root 2024-09-19T07:37:53.557 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-e[64090]: ignoring --setgroup ceph since I am not root 2024-09-19T07:37:53.557 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-mds[64094]: -- 172.21.15.189:0/3257448880 <== mon.2 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5599d9233860 con 0x5599d9211400 2024-09-19T07:37:53.557 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-e[64090]: starting mds.e at 2024-09-19T07:37:53.931 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T07:37:54.136 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:53 smithi159 ceph-mon[30472]: pgmap v521: 129 pgs: 129 active+clean; 1.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-19T07:37:54.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:53 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/461881330' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T07:37:54.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:53 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T07:37:54.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:53 smithi159 ceph-mon[30472]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T07:37:54.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:53 smithi159 ceph-mon[30472]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:37:54.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:53 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T07:37:54.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:53 smithi159 ceph-mon[30472]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T07:37:54.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:53 smithi159 ceph-mon[30472]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-19T07:37:54.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-mon[35335]: pgmap v521: 129 pgs: 129 active+clean; 1.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-19T07:37:54.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/461881330' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T07:37:54.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T07:37:54.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-mon[35335]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T07:37:54.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-mon[35335]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:37:54.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T07:37:54.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-mon[35335]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T07:37:54.287 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:53 smithi203 ceph-mon[35335]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-19T07:37:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-mon[34851]: pgmap v521: 129 pgs: 129 active+clean; 1.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-19T07:37:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/461881330' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T07:37:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T07:37:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-mon[34851]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T07:37:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-mon[34851]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:37:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T07:37:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-mon[34851]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T07:37:54.306 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:53 smithi189 ceph-mon[34851]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-19T07:37:54.332 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:54.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.962+0000 7fed61800640 1 -- 172.21.15.159:0/1466779750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed5c077040 msgr2=0x7fed5c112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:54.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.962+0000 7fed61800640 1 --2- 172.21.15.159:0/1466779750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed5c077040 0x7fed5c112330 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fed48009900 tx=0x7fed4802a3f0 comp rx=0 tx=0).stop 2024-09-19T07:37:54.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.962+0000 7fed61800640 1 -- 172.21.15.159:0/1466779750 shutdown_connections 2024-09-19T07:37:54.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.962+0000 7fed61800640 1 --2- 172.21.15.159:0/1466779750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed5c112870 0x7fed5c114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:54.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.962+0000 7fed61800640 1 --2- 172.21.15.159:0/1466779750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed5c077040 0x7fed5c112330 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:54.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.962+0000 7fed61800640 1 --2- 172.21.15.159:0/1466779750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed5c076720 0x7fed5c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:54.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.962+0000 7fed61800640 1 -- 172.21.15.159:0/1466779750 >> 172.21.15.159:0/1466779750 conn(0x7fed5c0ff910 msgr2=0x7fed5c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:54.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.962+0000 7fed61800640 1 -- 172.21.15.159:0/1466779750 shutdown_connections 2024-09-19T07:37:54.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.962+0000 7fed61800640 1 -- 172.21.15.159:0/1466779750 wait complete. 2024-09-19T07:37:54.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.963+0000 7fed61800640 1 Processor -- start 2024-09-19T07:37:54.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.963+0000 7fed61800640 1 -- start start 2024-09-19T07:37:54.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.964+0000 7fed61800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed5c076720 0x7fed5c1a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:54.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.964+0000 7fed61800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed5c077040 0x7fed5c1a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:54.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.964+0000 7fed61800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed5c112870 0x7fed5c19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:54.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.964+0000 7fed61800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c1183f0 con 0x7fed5c112870 2024-09-19T07:37:54.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.964+0000 7fed61800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c118270 con 0x7fed5c076720 2024-09-19T07:37:55.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.964+0000 7fed61800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c118570 con 0x7fed5c077040 2024-09-19T07:37:55.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.964+0000 7fed60e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed5c112870 0x7fed5c19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:55.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.964+0000 7fed5b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed5c077040 0x7fed5c1a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:55.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.964+0000 7fed60e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed5c112870 0x7fed5c19e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43020/0 (socket says 172.21.15.159:43020) 2024-09-19T07:37:55.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.964+0000 7fed60e00640 1 -- 172.21.15.159:0/2264024055 learned_addr learned my addr 172.21.15.159:0/2264024055 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:37:55.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.964+0000 7fed5b400640 1 --2- 172.21.15.159:0/2264024055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed5c077040 0x7fed5c1a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:50078/0 (socket says 172.21.15.159:50078) 2024-09-19T07:37:55.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.964+0000 7fed5be00640 1 --2- 172.21.15.159:0/2264024055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed5c076720 0x7fed5c1a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:55.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.965+0000 7fed5b400640 1 -- 172.21.15.159:0/2264024055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed5c076720 msgr2=0x7fed5c1a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:55.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.965+0000 7fed5b400640 1 --2- 172.21.15.159:0/2264024055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed5c076720 0x7fed5c1a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:55.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.965+0000 7fed5b400640 1 -- 172.21.15.159:0/2264024055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed5c112870 msgr2=0x7fed5c19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:55.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.965+0000 7fed5b400640 1 --2- 172.21.15.159:0/2264024055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed5c112870 0x7fed5c19e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:55.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.965+0000 7fed5b400640 1 -- 172.21.15.159:0/2264024055 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5c19ef70 con 0x7fed5c077040 2024-09-19T07:37:55.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.965+0000 7fed5be00640 1 --2- 172.21.15.159:0/2264024055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed5c076720 0x7fed5c1a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:37:55.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.965+0000 7fed60e00640 1 --2- 172.21.15.159:0/2264024055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed5c112870 0x7fed5c19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:37:55.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.965+0000 7fed5b400640 1 --2- 172.21.15.159:0/2264024055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed5c077040 0x7fed5c1a4b70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fed48005940 tx=0x7fed480040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:55.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.965+0000 7fed58c00640 1 -- 172.21.15.159:0/2264024055 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4803f030 con 0x7fed5c077040 2024-09-19T07:37:55.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.965+0000 7fed61800640 1 -- 172.21.15.159:0/2264024055 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5c19f200 con 0x7fed5c077040 2024-09-19T07:37:55.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.965+0000 7fed58c00640 1 -- 172.21.15.159:0/2264024055 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed480044c0 con 0x7fed5c077040 2024-09-19T07:37:55.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.965+0000 7fed58c00640 1 -- 172.21.15.159:0/2264024055 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48035070 con 0x7fed5c077040 2024-09-19T07:37:55.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.965+0000 7fed61800640 1 -- 172.21.15.159:0/2264024055 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5c1ab3e0 con 0x7fed5c077040 2024-09-19T07:37:55.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.966+0000 7fed61800640 1 -- 172.21.15.159:0/2264024055 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed5c077e60 con 0x7fed5c077040 2024-09-19T07:37:55.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.969+0000 7fed58c00640 1 -- 172.21.15.159:0/2264024055 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fed48042030 con 0x7fed5c077040 2024-09-19T07:37:55.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.970+0000 7fed58c00640 1 --2- 172.21.15.159:0/2264024055 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fed400a3240 0x7fed400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:55.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.970+0000 7fed58c00640 1 -- 172.21.15.159:0/2264024055 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9437+0+0 (secure 0 0 0) 0x7fed480f0340 con 0x7fed5c077040 2024-09-19T07:37:55.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.970+0000 7fed5be00640 1 --2- 172.21.15.159:0/2264024055 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fed400a3240 0x7fed400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:55.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.971+0000 7fed5be00640 1 --2- 172.21.15.159:0/2264024055 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fed400a3240 0x7fed400a5700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009fc0 tx=0x7fed4c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:55.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:54.972+0000 7fed58c00640 1 -- 172.21.15.159:0/2264024055 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fed480bc560 con 0x7fed5c077040 2024-09-19T07:37:55.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:55 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.203:6834/4160884699,v1:172.21.15.203:6835/4160884699] up:boot 2024-09-19T07:37:55.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:55 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.189:6834/2040532319,v1:172.21.15.189:6836/2040532319] up:boot 2024-09-19T07:37:55.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:55 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.189:6835/980693744,v1:172.21.15.189:6837/980693744] up:boot 2024-09-19T07:37:55.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:55 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.189:6838/2283767102,v1:172.21.15.189:6839/2283767102] up:boot 2024-09-19T07:37:55.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:55 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.203:6836/2426515746,v1:172.21.15.203:6837/2426515746] up:boot 2024-09-19T07:37:55.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:55 smithi159 ceph-mon[30472]: mds.? [v2:172.21.15.203:6838/3977172952,v1:172.21.15.203:6839/3977172952] up:boot 2024-09-19T07:37:55.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:55 smithi159 ceph-mon[30472]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-19T07:37:55.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:55 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:37:55.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:55 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:37:55.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:55 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:37:55.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:55 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:37:55.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:55 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:37:55.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:55 smithi159 ceph-mon[30472]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:37:55.138 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:55 smithi159 ceph-mon[30472]: pgmap v523: 129 pgs: 129 active+clean; 1.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-19T07:37:55.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:55.189+0000 7fed61800640 1 -- 172.21.15.159:0/2264024055 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fed5c076b00 con 0x7fed5c077040 2024-09-19T07:37:55.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:55 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.203:6834/4160884699,v1:172.21.15.203:6835/4160884699] up:boot 2024-09-19T07:37:55.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:55 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.189:6834/2040532319,v1:172.21.15.189:6836/2040532319] up:boot 2024-09-19T07:37:55.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:55 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.189:6835/980693744,v1:172.21.15.189:6837/980693744] up:boot 2024-09-19T07:37:55.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:55 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.189:6838/2283767102,v1:172.21.15.189:6839/2283767102] up:boot 2024-09-19T07:37:55.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:55 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.203:6836/2426515746,v1:172.21.15.203:6837/2426515746] up:boot 2024-09-19T07:37:55.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:55 smithi203 ceph-mon[35335]: mds.? [v2:172.21.15.203:6838/3977172952,v1:172.21.15.203:6839/3977172952] up:boot 2024-09-19T07:37:55.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:55 smithi203 ceph-mon[35335]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-19T07:37:55.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:55 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:37:55.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:55 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:37:55.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:55 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:37:55.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:55 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:37:55.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:55 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:37:55.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:55 smithi203 ceph-mon[35335]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:37:55.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:55 smithi203 ceph-mon[35335]: pgmap v523: 129 pgs: 129 active+clean; 1.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-19T07:37:55.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:55 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.203:6834/4160884699,v1:172.21.15.203:6835/4160884699] up:boot 2024-09-19T07:37:55.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:55 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.189:6834/2040532319,v1:172.21.15.189:6836/2040532319] up:boot 2024-09-19T07:37:55.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:55 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.189:6835/980693744,v1:172.21.15.189:6837/980693744] up:boot 2024-09-19T07:37:55.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:55 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.189:6838/2283767102,v1:172.21.15.189:6839/2283767102] up:boot 2024-09-19T07:37:55.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:55 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.203:6836/2426515746,v1:172.21.15.203:6837/2426515746] up:boot 2024-09-19T07:37:55.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:55 smithi189 ceph-mon[34851]: mds.? [v2:172.21.15.203:6838/3977172952,v1:172.21.15.203:6839/3977172952] up:boot 2024-09-19T07:37:55.556 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:55 smithi189 ceph-mon[34851]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-19T07:37:55.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:55 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:37:55.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:55 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:37:55.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:55 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:37:55.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:55 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:37:55.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:55 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:37:55.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:55 smithi189 ceph-mon[34851]: from='mgr.14148 172.21.15.159:0/765476257' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:37:55.557 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:55 smithi189 ceph-mon[34851]: pgmap v523: 129 pgs: 129 active+clean; 1.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-19T07:37:56.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:56.088+0000 7fed58c00640 1 -- 172.21.15.159:0/2264024055 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7fed480c0860 con 0x7fed5c077040 2024-09-19T07:37:56.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:56.091+0000 7fed61800640 1 -- 172.21.15.159:0/2264024055 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fed400a3240 msgr2=0x7fed400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:56.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:56.091+0000 7fed61800640 1 --2- 172.21.15.159:0/2264024055 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fed400a3240 0x7fed400a5700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009fc0 tx=0x7fed4c005f90 comp rx=0 tx=0).stop 2024-09-19T07:37:56.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:56.091+0000 7fed61800640 1 -- 172.21.15.159:0/2264024055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed5c077040 msgr2=0x7fed5c1a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:56.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:56.091+0000 7fed61800640 1 --2- 172.21.15.159:0/2264024055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed5c077040 0x7fed5c1a4b70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fed48005940 tx=0x7fed480040c0 comp rx=0 tx=0).stop 2024-09-19T07:37:56.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:56.091+0000 7fed61800640 1 -- 172.21.15.159:0/2264024055 shutdown_connections 2024-09-19T07:37:56.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:56.091+0000 7fed61800640 1 --2- 172.21.15.159:0/2264024055 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fed400a3240 0x7fed400a5700 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:56.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:56.091+0000 7fed61800640 1 --2- 172.21.15.159:0/2264024055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed5c112870 0x7fed5c19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:56.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:56.091+0000 7fed61800640 1 --2- 172.21.15.159:0/2264024055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed5c077040 0x7fed5c1a4b70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:56.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:56.091+0000 7fed61800640 1 --2- 172.21.15.159:0/2264024055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed5c076720 0x7fed5c1a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:56.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:56.091+0000 7fed61800640 1 -- 172.21.15.159:0/2264024055 >> 172.21.15.159:0/2264024055 conn(0x7fed5c0ff910 msgr2=0x7fed5c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:56.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:56.091+0000 7fed61800640 1 -- 172.21.15.159:0/2264024055 shutdown_connections 2024-09-19T07:37:56.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:56.091+0000 7fed61800640 1 -- 172.21.15.159:0/2264024055 wait complete. 2024-09-19T07:37:56.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:56 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/2264024055' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:37:56.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:56 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:37:56.457 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:56 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/2264024055' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:37:56.457 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:56 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:37:56.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:56 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/2264024055' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:37:56.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:56 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:37:57.053 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T07:37:57.386 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:57 smithi159 ceph-mon[30472]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T07:37:57.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:57 smithi159 ceph-mon[30472]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T07:37:57.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:57 smithi159 ceph-mon[30472]: Cluster is now healthy 2024-09-19T07:37:57.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:57 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T07:37:57.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:57 smithi159 ceph-mon[30472]: fsmap 12 up:standby 2024-09-19T07:37:57.387 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:57 smithi159 ceph-mon[30472]: pgmap v524: 129 pgs: 129 active+clean; 1.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T07:37:57.429 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:57.471 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:57 smithi189 ceph-mon[34851]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T07:37:57.471 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:57 smithi189 ceph-mon[34851]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T07:37:57.471 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:57 smithi189 ceph-mon[34851]: Cluster is now healthy 2024-09-19T07:37:57.471 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:57 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T07:37:57.471 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:57 smithi189 ceph-mon[34851]: fsmap 12 up:standby 2024-09-19T07:37:57.471 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:57 smithi189 ceph-mon[34851]: pgmap v524: 129 pgs: 129 active+clean; 1.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T07:37:57.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:57 smithi203 ceph-mon[35335]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T07:37:57.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:57 smithi203 ceph-mon[35335]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T07:37:57.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:57 smithi203 ceph-mon[35335]: Cluster is now healthy 2024-09-19T07:37:57.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:57 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T07:37:57.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:57 smithi203 ceph-mon[35335]: fsmap 12 up:standby 2024-09-19T07:37:57.536 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:57 smithi203 ceph-mon[35335]: pgmap v524: 129 pgs: 129 active+clean; 1.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-19T07:37:58.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.077+0000 7f6702e00640 1 -- 172.21.15.159:0/133948763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc101de0 msgr2=0x7f66fc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:58.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.077+0000 7f6702e00640 1 --2- 172.21.15.159:0/133948763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc101de0 0x7f66fc1021c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009900 tx=0x7f66ec02a3f0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.077+0000 7f6702e00640 1 -- 172.21.15.159:0/133948763 shutdown_connections 2024-09-19T07:37:58.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.077+0000 7f6702e00640 1 --2- 172.21.15.159:0/133948763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66fc106f10 0x7f66fc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.077+0000 7f6702e00640 1 --2- 172.21.15.159:0/133948763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66fc102790 0x7f66fc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.077+0000 7f6702e00640 1 --2- 172.21.15.159:0/133948763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc101de0 0x7f66fc1021c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.077+0000 7f6702e00640 1 -- 172.21.15.159:0/133948763 >> 172.21.15.159:0/133948763 conn(0x7f66fc0fd8c0 msgr2=0x7f66fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:58.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.077+0000 7f6702e00640 1 -- 172.21.15.159:0/133948763 shutdown_connections 2024-09-19T07:37:58.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.077+0000 7f6702e00640 1 -- 172.21.15.159:0/133948763 wait complete. 2024-09-19T07:37:58.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.078+0000 7f6702e00640 1 Processor -- start 2024-09-19T07:37:58.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.078+0000 7f6702e00640 1 -- start start 2024-09-19T07:37:58.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f6702e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc101de0 0x7f66fc19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:58.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f6702e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66fc102790 0x7f66fc19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:58.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f6702e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66fc106f10 0x7f66fc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:58.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f6702e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc111f30 con 0x7f66fc101de0 2024-09-19T07:37:58.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f6702e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc111db0 con 0x7f66fc102790 2024-09-19T07:37:58.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f6702e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc1120b0 con 0x7f66fc106f10 2024-09-19T07:37:58.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f66fbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc101de0 0x7f66fc19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:58.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f66f3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66fc102790 0x7f66fc19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:58.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f66fbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc101de0 0x7f66fc19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43044/0 (socket says 172.21.15.159:43044) 2024-09-19T07:37:58.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f66fbe00640 1 -- 172.21.15.159:0/3153871416 learned_addr learned my addr 172.21.15.159:0/3153871416 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:37:58.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f6700e00640 1 --2- 172.21.15.159:0/3153871416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66fc106f10 0x7f66fc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:58.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f66f3e00640 1 -- 172.21.15.159:0/3153871416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66fc106f10 msgr2=0x7f66fc19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:58.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f66f3e00640 1 --2- 172.21.15.159:0/3153871416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66fc106f10 0x7f66fc19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f66f3e00640 1 -- 172.21.15.159:0/3153871416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc101de0 msgr2=0x7f66fc19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:58.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f66f3e00640 1 --2- 172.21.15.159:0/3153871416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc101de0 0x7f66fc19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f66f3e00640 1 -- 172.21.15.159:0/3153871416 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66fc19f620 con 0x7f66fc102790 2024-09-19T07:37:58.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f6700e00640 1 --2- 172.21.15.159:0/3153871416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66fc106f10 0x7f66fc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:58.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.079+0000 7f66fbe00640 1 --2- 172.21.15.159:0/3153871416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc101de0 0x7f66fc19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:37:58.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.080+0000 7f66f3e00640 1 --2- 172.21.15.159:0/3153871416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66fc102790 0x7f66fc19abb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f66e400e9f0 tx=0x7f66e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:58.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.080+0000 7f66f9600640 1 -- 172.21.15.159:0/3153871416 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e400ccc0 con 0x7f66fc102790 2024-09-19T07:37:58.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.080+0000 7f66f9600640 1 -- 172.21.15.159:0/3153871416 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66e4010910 con 0x7f66fc102790 2024-09-19T07:37:58.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.080+0000 7f6702e00640 1 -- 172.21.15.159:0/3153871416 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66fc19f910 con 0x7f66fc102790 2024-09-19T07:37:58.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.080+0000 7f66f9600640 1 -- 172.21.15.159:0/3153871416 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4018960 con 0x7f66fc102790 2024-09-19T07:37:58.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.080+0000 7f6702e00640 1 -- 172.21.15.159:0/3153871416 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66fc1a71f0 con 0x7f66fc102790 2024-09-19T07:37:58.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.081+0000 7f6702e00640 1 -- 172.21.15.159:0/3153871416 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66fc1021c0 con 0x7f66fc102790 2024-09-19T07:37:58.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.083+0000 7f66f9600640 1 -- 172.21.15.159:0/3153871416 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f66e4010ac0 con 0x7f66fc102790 2024-09-19T07:37:58.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.084+0000 7f66f9600640 1 --2- 172.21.15.159:0/3153871416 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f66e00a3240 0x7f66e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:58.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.084+0000 7f66f9600640 1 -- 172.21.15.159:0/3153871416 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9437+0+0 (secure 0 0 0) 0x7f66e40d0260 con 0x7f66fc102790 2024-09-19T07:37:58.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.085+0000 7f66fbe00640 1 --2- 172.21.15.159:0/3153871416 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f66e00a3240 0x7f66e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:58.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.085+0000 7f66fbe00640 1 --2- 172.21.15.159:0/3153871416 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f66e00a3240 0x7f66e00a5700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0096c0 tx=0x7f66ec005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:58.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.088+0000 7f66f9600640 1 -- 172.21.15.159:0/3153871416 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f66e409ca70 con 0x7f66fc102790 2024-09-19T07:37:58.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.266+0000 7f6702e00640 1 -- 172.21.15.159:0/3153871416 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f66fc1a7520 con 0x7f66fc102790 2024-09-19T07:37:58.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.323+0000 7f66f9600640 1 -- 172.21.15.159:0/3153871416 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7f66e409c370 con 0x7f66fc102790 2024-09-19T07:37:58.324 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' removed 2024-09-19T07:37:58.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.326+0000 7f6702e00640 1 -- 172.21.15.159:0/3153871416 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f66e00a3240 msgr2=0x7f66e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:58.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.326+0000 7f6702e00640 1 --2- 172.21.15.159:0/3153871416 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f66e00a3240 0x7f66e00a5700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0096c0 tx=0x7f66ec005a20 comp rx=0 tx=0).stop 2024-09-19T07:37:58.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.326+0000 7f6702e00640 1 -- 172.21.15.159:0/3153871416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66fc102790 msgr2=0x7f66fc19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:58.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.326+0000 7f6702e00640 1 --2- 172.21.15.159:0/3153871416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66fc102790 0x7f66fc19abb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f66e400e9f0 tx=0x7f66e400eeb0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.327+0000 7f6702e00640 1 -- 172.21.15.159:0/3153871416 shutdown_connections 2024-09-19T07:37:58.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.327+0000 7f6702e00640 1 --2- 172.21.15.159:0/3153871416 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7f66e00a3240 0x7f66e00a5700 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.327+0000 7f6702e00640 1 --2- 172.21.15.159:0/3153871416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66fc106f10 0x7f66fc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.327+0000 7f6702e00640 1 --2- 172.21.15.159:0/3153871416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66fc102790 0x7f66fc19abb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.327+0000 7f6702e00640 1 --2- 172.21.15.159:0/3153871416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc101de0 0x7f66fc19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.327+0000 7f6702e00640 1 -- 172.21.15.159:0/3153871416 >> 172.21.15.159:0/3153871416 conn(0x7f66fc0fd8c0 msgr2=0x7f66fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:58.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.327+0000 7f6702e00640 1 -- 172.21.15.159:0/3153871416 shutdown_connections 2024-09-19T07:37:58.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:37:58.327+0000 7f6702e00640 1 -- 172.21.15.159:0/3153871416 wait complete. 2024-09-19T07:37:58.636 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:58 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/3153871416' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:37:58.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:58 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:37:58.696 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:58 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/3153871416' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:37:58.696 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:58 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:37:58.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:58 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/3153871416' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:37:58.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:58 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:37:59.218 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T07:37:59.589 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:59.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:59 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:37:59.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:59 smithi159 ceph-mon[30472]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T07:37:59.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:37:59 smithi159 ceph-mon[30472]: pgmap v526: 97 pgs: 97 active+clean; 577 KiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:37:59.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:59 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:37:59.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:59 smithi203 ceph-mon[35335]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T07:37:59.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:37:59 smithi203 ceph-mon[35335]: pgmap v526: 97 pgs: 97 active+clean; 577 KiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:37:59.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:59 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:37:59.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:59 smithi189 ceph-mon[34851]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T07:37:59.806 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:37:59 smithi189 ceph-mon[34851]: pgmap v526: 97 pgs: 97 active+clean; 577 KiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:38:00.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.389+0000 7fa362c00640 1 -- 172.21.15.159:0/4285887348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa35c074e30 msgr2=0x7fa35c10df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:00.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.389+0000 7fa362c00640 1 --2- 172.21.15.159:0/4285887348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa35c074e30 0x7fa35c10df00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa344009860 tx=0x7fa34402a3d0 comp rx=0 tx=0).stop 2024-09-19T07:38:00.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.389+0000 7fa362c00640 1 -- 172.21.15.159:0/4285887348 shutdown_connections 2024-09-19T07:38:00.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.389+0000 7fa362c00640 1 --2- 172.21.15.159:0/4285887348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa35c10e440 0x7fa35c110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:00.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.389+0000 7fa362c00640 1 --2- 172.21.15.159:0/4285887348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa35c074e30 0x7fa35c10df00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:00.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.389+0000 7fa362c00640 1 --2- 172.21.15.159:0/4285887348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa35c074510 0x7fa35c0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:00.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.389+0000 7fa362c00640 1 -- 172.21.15.159:0/4285887348 >> 172.21.15.159:0/4285887348 conn(0x7fa35c0fd8c0 msgr2=0x7fa35c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:00.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.389+0000 7fa362c00640 1 -- 172.21.15.159:0/4285887348 shutdown_connections 2024-09-19T07:38:00.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.390+0000 7fa362c00640 1 -- 172.21.15.159:0/4285887348 wait complete. 2024-09-19T07:38:00.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.390+0000 7fa362c00640 1 Processor -- start 2024-09-19T07:38:00.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.391+0000 7fa362c00640 1 -- start start 2024-09-19T07:38:00.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.391+0000 7fa362c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa35c074510 0x7fa35c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:00.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.391+0000 7fa362c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa35c074e30 0x7fa35c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:00.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.391+0000 7fa362c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa35c10e440 0x7fa35c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:00.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.391+0000 7fa362c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c114140 con 0x7fa35c074e30 2024-09-19T07:38:00.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.391+0000 7fa362c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c113fc0 con 0x7fa35c10e440 2024-09-19T07:38:00.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.391+0000 7fa362c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1142c0 con 0x7fa35c074510 2024-09-19T07:38:00.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.391+0000 7fa353400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa35c074e30 0x7fa35c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:00.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.391+0000 7fa35be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa35c074510 0x7fa35c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:00.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.391+0000 7fa35be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa35c074510 0x7fa35c19a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.159:50108/0 (socket says 172.21.15.159:50108) 2024-09-19T07:38:00.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.391+0000 7fa353400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa35c074e30 0x7fa35c19ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43070/0 (socket says 172.21.15.159:43070) 2024-09-19T07:38:00.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.391+0000 7fa35be00640 1 -- 172.21.15.159:0/1407517932 learned_addr learned my addr 172.21.15.159:0/1407517932 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:38:00.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.392+0000 7fa360c00640 1 --2- 172.21.15.159:0/1407517932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa35c10e440 0x7fa35c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:00.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.392+0000 7fa35be00640 1 -- 172.21.15.159:0/1407517932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa35c10e440 msgr2=0x7fa35c19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:00.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.392+0000 7fa35be00640 1 --2- 172.21.15.159:0/1407517932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa35c10e440 0x7fa35c19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:00.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.392+0000 7fa35be00640 1 -- 172.21.15.159:0/1407517932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa35c074e30 msgr2=0x7fa35c19ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:00.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.392+0000 7fa35be00640 1 --2- 172.21.15.159:0/1407517932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa35c074e30 0x7fa35c19ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:00.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.392+0000 7fa35be00640 1 -- 172.21.15.159:0/1407517932 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c19f6d0 con 0x7fa35c074510 2024-09-19T07:38:00.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.392+0000 7fa35be00640 1 --2- 172.21.15.159:0/1407517932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa35c074510 0x7fa35c19a720 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00e9f0 tx=0x7fa34c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:00.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.392+0000 7fa359600640 1 -- 172.21.15.159:0/1407517932 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c00cd80 con 0x7fa35c074510 2024-09-19T07:38:00.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.392+0000 7fa362c00640 1 -- 172.21.15.159:0/1407517932 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35c19f9c0 con 0x7fa35c074510 2024-09-19T07:38:00.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.392+0000 7fa359600640 1 -- 172.21.15.159:0/1407517932 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa34c010910 con 0x7fa35c074510 2024-09-19T07:38:00.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.392+0000 7fa359600640 1 -- 172.21.15.159:0/1407517932 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0188f0 con 0x7fa35c074510 2024-09-19T07:38:00.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.393+0000 7fa362c00640 1 -- 172.21.15.159:0/1407517932 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35c1a72a0 con 0x7fa35c074510 2024-09-19T07:38:00.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.394+0000 7fa362c00640 1 -- 172.21.15.159:0/1407517932 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35c0748f0 con 0x7fa35c074510 2024-09-19T07:38:00.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.396+0000 7fa359600640 1 -- 172.21.15.159:0/1407517932 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fa34c010ac0 con 0x7fa35c074510 2024-09-19T07:38:00.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.397+0000 7fa359600640 1 --2- 172.21.15.159:0/1407517932 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa3400a3240 0x7fa3400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:00.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.397+0000 7fa359600640 1 -- 172.21.15.159:0/1407517932 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8883+0+0 (secure 0 0 0) 0x7fa34c0d0330 con 0x7fa35c074510 2024-09-19T07:38:00.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.397+0000 7fa353400640 1 --2- 172.21.15.159:0/1407517932 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa3400a3240 0x7fa3400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:00.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.398+0000 7fa353400640 1 --2- 172.21.15.159:0/1407517932 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa3400a3240 0x7fa3400a5700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0fff90 tx=0x7fa3440094d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:00.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.399+0000 7fa359600640 1 -- 172.21.15.159:0/1407517932 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa34c09c770 con 0x7fa35c074510 2024-09-19T07:38:00.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.581+0000 7fa362c00640 1 -- 172.21.15.159:0/1407517932 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa35c1095f0 con 0x7fa35c074510 2024-09-19T07:38:00.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.639+0000 7fa359600640 1 -- 172.21.15.159:0/1407517932 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7fa34c0a0a70 con 0x7fa35c074510 2024-09-19T07:38:00.639 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' removed 2024-09-19T07:38:00.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.642+0000 7fa362c00640 1 -- 172.21.15.159:0/1407517932 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa3400a3240 msgr2=0x7fa3400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:00.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.642+0000 7fa362c00640 1 --2- 172.21.15.159:0/1407517932 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa3400a3240 0x7fa3400a5700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0fff90 tx=0x7fa3440094d0 comp rx=0 tx=0).stop 2024-09-19T07:38:00.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.642+0000 7fa362c00640 1 -- 172.21.15.159:0/1407517932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa35c074510 msgr2=0x7fa35c19a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:00.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.642+0000 7fa362c00640 1 --2- 172.21.15.159:0/1407517932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa35c074510 0x7fa35c19a720 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00e9f0 tx=0x7fa34c00eeb0 comp rx=0 tx=0).stop 2024-09-19T07:38:00.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.643+0000 7fa362c00640 1 -- 172.21.15.159:0/1407517932 shutdown_connections 2024-09-19T07:38:00.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.643+0000 7fa362c00640 1 --2- 172.21.15.159:0/1407517932 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fa3400a3240 0x7fa3400a5700 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:00.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.643+0000 7fa362c00640 1 --2- 172.21.15.159:0/1407517932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa35c10e440 0x7fa35c19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:00.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.643+0000 7fa362c00640 1 --2- 172.21.15.159:0/1407517932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa35c074e30 0x7fa35c19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:00.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.643+0000 7fa362c00640 1 --2- 172.21.15.159:0/1407517932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa35c074510 0x7fa35c19a720 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:00.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.643+0000 7fa362c00640 1 -- 172.21.15.159:0/1407517932 >> 172.21.15.159:0/1407517932 conn(0x7fa35c0fd8c0 msgr2=0x7fa35c0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:00.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.644+0000 7fa362c00640 1 -- 172.21.15.159:0/1407517932 shutdown_connections 2024-09-19T07:38:00.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:00.644+0000 7fa362c00640 1 -- 172.21.15.159:0/1407517932 wait complete. 2024-09-19T07:38:00.931 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:38:00 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1407517932' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:38:00.932 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:38:00 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:38:01.021 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:38:00 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1407517932' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:38:01.021 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:38:00 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:38:01.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:38:00 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1407517932' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:38:01.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:38:00 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:38:01.551 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid c9d18048-7657-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-19T07:38:01.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:38:01 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:38:01.788 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:38:01 smithi203 ceph-mon[35335]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T07:38:01.788 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:38:01 smithi203 ceph-mon[35335]: pgmap v528: 33 pgs: 33 active+clean; 577 KiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:38:01.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:38:01 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:38:01.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:38:01 smithi159 ceph-mon[30472]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T07:38:01.887 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:38:01 smithi159 ceph-mon[30472]: pgmap v528: 33 pgs: 33 active+clean; 577 KiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:38:01.926 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:02.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:38:01 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:38:02.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:38:01 smithi189 ceph-mon[34851]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T07:38:02.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:38:01 smithi189 ceph-mon[34851]: pgmap v528: 33 pgs: 33 active+clean; 577 KiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:38:02.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.507+0000 7fc1d9a00640 1 -- 172.21.15.159:0/4090807735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d41047e0 msgr2=0x7fc1d4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:02.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.507+0000 7fc1d9a00640 1 --2- 172.21.15.159:0/4090807735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d41047e0 0x7fc1d4108830 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009900 tx=0x7fc1c002a410 comp rx=0 tx=0).stop 2024-09-19T07:38:02.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.507+0000 7fc1d9a00640 1 -- 172.21.15.159:0/4090807735 shutdown_connections 2024-09-19T07:38:02.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.507+0000 7fc1d9a00640 1 --2- 172.21.15.159:0/4090807735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1d4108f60 0x7fc1d4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.507+0000 7fc1d9a00640 1 --2- 172.21.15.159:0/4090807735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d41047e0 0x7fc1d4108830 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.507+0000 7fc1d9a00640 1 --2- 172.21.15.159:0/4090807735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1d4103e30 0x7fc1d4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.507+0000 7fc1d9a00640 1 -- 172.21.15.159:0/4090807735 >> 172.21.15.159:0/4090807735 conn(0x7fc1d40ff8d0 msgr2=0x7fc1d4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:02.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.507+0000 7fc1d9a00640 1 -- 172.21.15.159:0/4090807735 shutdown_connections 2024-09-19T07:38:02.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.507+0000 7fc1d9a00640 1 -- 172.21.15.159:0/4090807735 wait complete. 2024-09-19T07:38:02.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.508+0000 7fc1d9a00640 1 Processor -- start 2024-09-19T07:38:02.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.509+0000 7fc1d9a00640 1 -- start start 2024-09-19T07:38:02.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.509+0000 7fc1d9a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d4103e30 0x7fc1d419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:02.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.509+0000 7fc1d9a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1d41047e0 0x7fc1d419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:02.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.509+0000 7fc1d9a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1d4108f60 0x7fc1d419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:02.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.509+0000 7fc1d9a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d4113fc0 con 0x7fc1d4103e30 2024-09-19T07:38:02.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.509+0000 7fc1d9a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d4113e40 con 0x7fc1d41047e0 2024-09-19T07:38:02.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.509+0000 7fc1d9a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d4114140 con 0x7fc1d4108f60 2024-09-19T07:38:02.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.509+0000 7fc1d3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d4103e30 0x7fc1d419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:02.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.509+0000 7fc1d3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d4103e30 0x7fc1d419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43216/0 (socket says 172.21.15.159:43216) 2024-09-19T07:38:02.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.509+0000 7fc1d3e00640 1 -- 172.21.15.159:0/1283693851 learned_addr learned my addr 172.21.15.159:0/1283693851 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-19T07:38:02.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.509+0000 7fc1d3400640 1 --2- 172.21.15.159:0/1283693851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1d41047e0 0x7fc1d419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:02.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.510+0000 7fc1d9000640 1 --2- 172.21.15.159:0/1283693851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1d4108f60 0x7fc1d419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:02.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.510+0000 7fc1d3400640 1 -- 172.21.15.159:0/1283693851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1d4108f60 msgr2=0x7fc1d419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:02.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.510+0000 7fc1d3400640 1 --2- 172.21.15.159:0/1283693851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1d4108f60 0x7fc1d419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.510+0000 7fc1d3400640 1 -- 172.21.15.159:0/1283693851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d4103e30 msgr2=0x7fc1d419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:02.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.510+0000 7fc1d3400640 1 --2- 172.21.15.159:0/1283693851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d4103e30 0x7fc1d419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.510+0000 7fc1d3400640 1 -- 172.21.15.159:0/1283693851 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d419f450 con 0x7fc1d41047e0 2024-09-19T07:38:02.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.510+0000 7fc1d9000640 1 --2- 172.21.15.159:0/1283693851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1d4108f60 0x7fc1d419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:02.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.510+0000 7fc1d3400640 1 --2- 172.21.15.159:0/1283693851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1d41047e0 0x7fc1d419a9e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0005940 tx=0x7fc1c0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:02.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.511+0000 7fc1d0c00640 1 -- 172.21.15.159:0/1283693851 <== mon.2 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c003f030 con 0x7fc1d41047e0 2024-09-19T07:38:02.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.511+0000 7fc1d9a00640 1 -- 172.21.15.159:0/1283693851 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d419f6e0 con 0x7fc1d41047e0 2024-09-19T07:38:02.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.511+0000 7fc1d0c00640 1 -- 172.21.15.159:0/1283693851 <== mon.2 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1c0003ad0 con 0x7fc1d41047e0 2024-09-19T07:38:02.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.511+0000 7fc1d0c00640 1 -- 172.21.15.159:0/1283693851 <== mon.2 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0035070 con 0x7fc1d41047e0 2024-09-19T07:38:02.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.511+0000 7fc1d9a00640 1 -- 172.21.15.159:0/1283693851 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d41a7020 con 0x7fc1d41047e0 2024-09-19T07:38:02.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.512+0000 7fc1d9a00640 1 -- 172.21.15.159:0/1283693851 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d4104210 con 0x7fc1d41047e0 2024-09-19T07:38:02.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.514+0000 7fc1d0c00640 1 -- 172.21.15.159:0/1283693851 <== mon.2 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fc1c0042030 con 0x7fc1d41047e0 2024-09-19T07:38:02.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.515+0000 7fc1d0c00640 1 --2- 172.21.15.159:0/1283693851 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc1b80a3240 0x7fc1b80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:02.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.515+0000 7fc1d0c00640 1 -- 172.21.15.159:0/1283693851 <== mon.2 v2:172.21.15.189:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8324+0+0 (secure 0 0 0) 0x7fc1c00f0350 con 0x7fc1d41047e0 2024-09-19T07:38:02.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.515+0000 7fc1d3e00640 1 --2- 172.21.15.159:0/1283693851 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc1b80a3240 0x7fc1b80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:02.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.516+0000 7fc1d3e00640 1 --2- 172.21.15.159:0/1283693851 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc1b80a3240 0x7fc1b80a5700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009c90 tx=0x7fc1c4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:02.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.517+0000 7fc1d0c00640 1 -- 172.21.15.159:0/1283693851 <== mon.2 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc1c00bcfb0 con 0x7fc1d41047e0 2024-09-19T07:38:02.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.694+0000 7fc1d9a00640 1 -- 172.21.15.159:0/1283693851 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc1d419b800 con 0x7fc1d41047e0 2024-09-19T07:38:02.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.752+0000 7fc1d0c00640 1 -- 172.21.15.159:0/1283693851 <== mon.2 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v104) ==== 178+0+0 (secure 0 0 0) 0x7fc1c00bc8b0 con 0x7fc1d41047e0 2024-09-19T07:38:02.752 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data_ec' removed 2024-09-19T07:38:02.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.755+0000 7fc1d9a00640 1 -- 172.21.15.159:0/1283693851 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc1b80a3240 msgr2=0x7fc1b80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:02.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.755+0000 7fc1d9a00640 1 --2- 172.21.15.159:0/1283693851 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc1b80a3240 0x7fc1b80a5700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009c90 tx=0x7fc1c4005f90 comp rx=0 tx=0).stop 2024-09-19T07:38:02.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.755+0000 7fc1d9a00640 1 -- 172.21.15.159:0/1283693851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1d41047e0 msgr2=0x7fc1d419a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:02.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.755+0000 7fc1d9a00640 1 --2- 172.21.15.159:0/1283693851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1d41047e0 0x7fc1d419a9e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0005940 tx=0x7fc1c0004270 comp rx=0 tx=0).stop 2024-09-19T07:38:02.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.756+0000 7fc1d9a00640 1 -- 172.21.15.159:0/1283693851 shutdown_connections 2024-09-19T07:38:02.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.756+0000 7fc1d9a00640 1 --2- 172.21.15.159:0/1283693851 >> [v2:172.21.15.159:6800/4243905339,v1:172.21.15.159:6801/4243905339] conn(0x7fc1b80a3240 0x7fc1b80a5700 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.756+0000 7fc1d9a00640 1 --2- 172.21.15.159:0/1283693851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1d4108f60 0x7fc1d419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.756+0000 7fc1d9a00640 1 --2- 172.21.15.159:0/1283693851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1d41047e0 0x7fc1d419a9e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.756+0000 7fc1d9a00640 1 --2- 172.21.15.159:0/1283693851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d4103e30 0x7fc1d419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.756+0000 7fc1d9a00640 1 -- 172.21.15.159:0/1283693851 >> 172.21.15.159:0/1283693851 conn(0x7fc1d40ff8d0 msgr2=0x7fc1d4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:02.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.756+0000 7fc1d9a00640 1 -- 172.21.15.159:0/1283693851 shutdown_connections 2024-09-19T07:38:02.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-19T07:38:02.756+0000 7fc1d9a00640 1 -- 172.21.15.159:0/1283693851 wait complete. 2024-09-19T07:38:03.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:38:02 smithi203 ceph-mon[35335]: from='client.? 172.21.15.159:0/1283693851' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:38:03.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:38:02 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:38:03.055 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:38:02 smithi189 ceph-mon[34851]: from='client.? 172.21.15.159:0/1283693851' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:38:03.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:38:02 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:38:03.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:38:02 smithi159 ceph-mon[30472]: from='client.? 172.21.15.159:0/1283693851' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:38:03.137 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:38:02 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:38:03.712 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:38:03.773 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:38:03.807 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:38:03.842 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T07:38:03.843 DEBUG:teuthology.orchestra.run.smithi159:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T07:38:03.878 DEBUG:teuthology.orchestra.run.smithi189:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T07:38:03.896 DEBUG:teuthology.orchestra.run.smithi203:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T07:38:03.914 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T07:38:03.915 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-19T07:38:03.915 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mon.a 2024-09-19T07:38:03.967 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:38:03 smithi159 ceph-mon[30472]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:38:03.968 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:38:03 smithi159 ceph-mon[30472]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T07:38:03.968 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:38:03 smithi159 ceph-mon[30472]: pgmap v530: 1 pgs: 1 active+clean; 577 KiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:38:04.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:38:03 smithi203 ceph-mon[35335]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:38:04.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:38:03 smithi203 ceph-mon[35335]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T07:38:04.036 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:38:03 smithi203 ceph-mon[35335]: pgmap v530: 1 pgs: 1 active+clean; 577 KiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:38:04.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:38:03 smithi189 ceph-mon[34851]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:38:04.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:38:03 smithi189 ceph-mon[34851]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T07:38:04.056 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:38:03 smithi189 ceph-mon[34851]: pgmap v530: 1 pgs: 1 active+clean; 577 KiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:38:04.372 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:38:03 smithi159 systemd[1]: Stopping Ceph mon.a for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:38:04.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:38:04 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-a[30468]: 2024-09-19T07:38:04.371+0000 7f5fa4c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:38:04.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:38:04 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-a[30468]: 2024-09-19T07:38:04.371+0000 7f5fa4c00640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-19T07:38:04.637 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:38:04 smithi159 podman[87325]: 2024-09-19 07:38:04.433673568 +0000 UTC m=+0.234339953 container died 37afdc5581e3e9dde80d4424cafe97f73df3462a265d8296a8ddb3d59549ac06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T07:38:05.396 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:38:05 smithi159 podman[87325]: 2024-09-19 07:38:05.143651389 +0000 UTC m=+0.944317765 container cleanup 37afdc5581e3e9dde80d4424cafe97f73df3462a265d8296a8ddb3d59549ac06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T07:38:05.396 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:38:05 smithi159 bash[87325]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-a 2024-09-19T07:38:05.396 INFO:journalctl@ceph.mon.a.smithi159.stdout:Sep 19 07:38:05 smithi159 podman[87336]: 2024-09-19 07:38:05.39542715 +0000 UTC m=+0.948964413 container remove 37afdc5581e3e9dde80d4424cafe97f73df3462a265d8296a8ddb3d59549ac06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T07:38:05.683 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mon.a.service' 2024-09-19T07:38:05.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:38:05.788 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-19T07:38:05.788 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-19T07:38:05.788 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mon.b 2024-09-19T07:38:06.112 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:38:05 smithi189 systemd[1]: Stopping Ceph mon.b for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:38:06.455 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:38:06 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-b[34847]: 2024-09-19T07:38:06.110+0000 7fa695388640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:38:06.455 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:38:06 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-b[34847]: 2024-09-19T07:38:06.110+0000 7fa695388640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-19T07:38:06.455 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:38:06 smithi189 podman[67608]: 2024-09-19 07:38:06.132033768 +0000 UTC m=+0.139708012 container died e3b690676dfcb4ef321000fdd584730a6dda23ed638567563faa90e0dc3e0714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2) 2024-09-19T07:38:06.788 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:38:06 smithi189 podman[67608]: 2024-09-19 07:38:06.554136874 +0000 UTC m=+0.561811115 container cleanup e3b690676dfcb4ef321000fdd584730a6dda23ed638567563faa90e0dc3e0714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-b, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T07:38:06.788 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:38:06 smithi189 bash[67608]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-b 2024-09-19T07:38:07.039 INFO:journalctl@ceph.mon.b.smithi189.stdout:Sep 19 07:38:06 smithi189 podman[67620]: 2024-09-19 07:38:06.788978555 +0000 UTC m=+0.654443378 container remove e3b690676dfcb4ef321000fdd584730a6dda23ed638567563faa90e0dc3e0714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:38:07.041 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mon.b.service' 2024-09-19T07:38:07.097 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:38:07.097 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-19T07:38:07.098 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-19T07:38:07.098 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mon.c 2024-09-19T07:38:07.450 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:38:07 smithi203 systemd[1]: Stopping Ceph mon.c for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:38:07.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:38:07 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-c[35331]: 2024-09-19T07:38:07.449+0000 7f27de9da640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:38:07.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:38:07 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-c[35331]: 2024-09-19T07:38:07.450+0000 7f27de9da640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-19T07:38:07.786 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:38:07 smithi203 podman[68799]: 2024-09-19 07:38:07.613573009 +0000 UTC m=+0.311103601 container died 9bc3c2e8fa4134a948a51e755c242100c092b08dc0eb0c0339ba7269a417167a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T07:38:08.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:38:08 smithi203 podman[68799]: 2024-09-19 07:38:08.033360893 +0000 UTC m=+0.730891493 container cleanup 9bc3c2e8fa4134a948a51e755c242100c092b08dc0eb0c0339ba7269a417167a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-c, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-19T07:38:08.286 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:38:08 smithi203 bash[68799]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-c 2024-09-19T07:38:08.585 INFO:journalctl@ceph.mon.c.smithi203.stdout:Sep 19 07:38:08 smithi203 podman[68811]: 2024-09-19 07:38:08.293695617 +0000 UTC m=+0.684230139 container remove 9bc3c2e8fa4134a948a51e755c242100c092b08dc0eb0c0339ba7269a417167a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mon-c, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-19T07:38:08.605 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mon.c.service' 2024-09-19T07:38:08.653 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:38:08.653 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-19T07:38:08.653 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-19T07:38:08.654 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mgr.x 2024-09-19T07:38:09.086 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:38:08 smithi159 systemd[1]: Stopping Ceph mgr.x for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:38:09.387 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:38:09 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:38:09.084+0000 7f08e2e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:38:09.387 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:38:09 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x[30696]: 2024-09-19T07:38:09.084+0000 7f08e2e00640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-19T07:38:09.387 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:38:09 smithi159 podman[87442]: 2024-09-19 07:38:09.168991481 +0000 UTC m=+0.228793660 container died 69b92e6f3945fd74c1b2acc742cf7d78781715e0fb36f654e6e56351f3fac875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T07:38:10.134 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:38:09 smithi159 podman[87442]: 2024-09-19 07:38:09.799634245 +0000 UTC m=+0.859436429 container cleanup 69b92e6f3945fd74c1b2acc742cf7d78781715e0fb36f654e6e56351f3fac875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4) 2024-09-19T07:38:10.135 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:38:09 smithi159 bash[87442]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x 2024-09-19T07:38:10.386 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Sep 19 07:38:10 smithi159 podman[87454]: 2024-09-19 07:38:10.134795775 +0000 UTC m=+0.956362037 container remove 69b92e6f3945fd74c1b2acc742cf7d78781715e0fb36f654e6e56351f3fac875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-19T07:38:10.440 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-19T07:38:10.514 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:38:10.514 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-19T07:38:10.515 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-19T07:38:10.515 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mgr.y 2024-09-19T07:38:10.806 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:38:10 smithi189 systemd[1]: Stopping Ceph mgr.y for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:38:11.608 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:38:10 smithi189 podman[67730]: 2024-09-19 07:38:10.863907565 +0000 UTC m=+0.154120429 container died 315155a80885975d05fd80fe26702a2b33acb6d8530ac4eabf9fbc868155cc93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T07:38:11.608 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:38:11 smithi189 podman[67730]: 2024-09-19 07:38:11.276700083 +0000 UTC m=+0.566912946 container cleanup 315155a80885975d05fd80fe26702a2b33acb6d8530ac4eabf9fbc868155cc93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T07:38:11.608 INFO:journalctl@ceph.mgr.y.smithi189.stdout:Sep 19 07:38:11 smithi189 bash[67730]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-y 2024-09-19T07:38:11.715 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-19T07:38:11.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:38:11.750 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-19T07:38:11.750 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-19T07:38:11.750 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mgr.z 2024-09-19T07:38:12.036 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:38:11 smithi203 systemd[1]: Stopping Ceph mgr.z for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:38:12.481 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:38:12 smithi203 podman[68924]: 2024-09-19 07:38:12.092889549 +0000 UTC m=+0.151318942 container died 92614ce544d13c81f1438ca2acf5243c604c2610dd24e508390160e574fb4216 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:38:12.786 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:38:12 smithi203 podman[68924]: 2024-09-19 07:38:12.581126859 +0000 UTC m=+0.639556250 container cleanup 92614ce544d13c81f1438ca2acf5243c604c2610dd24e508390160e574fb4216 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:38:12.786 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Sep 19 07:38:12 smithi203 bash[68924]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mgr-z 2024-09-19T07:38:13.011 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-19T07:38:13.045 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:38:13.045 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-19T07:38:13.046 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T07:38:13.046 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.0 2024-09-19T07:38:13.387 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:38:13 smithi159 systemd[1]: Stopping Ceph osd.0 for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:38:13.887 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:38:13 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-0[44311]: 2024-09-19T07:38:13.472+0000 7f47e7200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:38:13.887 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:38:13 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-0[44311]: 2024-09-19T07:38:13.472+0000 7f47e7200640 -1 osd.0 104 *** Got signal Terminated *** 2024-09-19T07:38:13.887 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:38:13 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-0[44311]: 2024-09-19T07:38:13.472+0000 7f47e7200640 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:38:19.136 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:38:18 smithi159 podman[87561]: 2024-09-19 07:38:18.87216702 +0000 UTC m=+5.541572291 container died e15ba3bfdc4b0e527cc1e39d8ddaa0778db3b629d204c052908226be9e9a1bfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-0, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T07:38:19.505 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:38:19 smithi159 podman[87561]: 2024-09-19 07:38:19.479533108 +0000 UTC m=+6.148938365 container cleanup e15ba3bfdc4b0e527cc1e39d8ddaa0778db3b629d204c052908226be9e9a1bfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:38:19.505 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:38:19 smithi159 bash[87561]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-0 2024-09-19T07:38:20.136 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:38:19 smithi159 podman[87577]: 2024-09-19 07:38:19.756268139 +0000 UTC m=+0.875689364 container remove e15ba3bfdc4b0e527cc1e39d8ddaa0778db3b629d204c052908226be9e9a1bfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-0, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:38:20.621 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:38:20 smithi159 podman[87667]: 2024-09-19 07:38:20.209808297 +0000 UTC m=+0.027212898 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:38:20.621 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:38:20 smithi159 podman[87667]: 2024-09-19 07:38:20.332351511 +0000 UTC m=+0.149756110 container create 56767aff7488a33c6c133a4cc4cef7c8d456ff5919faef29281c4f7955d62470 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS) 2024-09-19T07:38:20.877 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:38:20 smithi159 podman[87667]: 2024-09-19 07:38:20.618639335 +0000 UTC m=+0.436043950 container init 56767aff7488a33c6c133a4cc4cef7c8d456ff5919faef29281c4f7955d62470 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-19T07:38:20.877 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:38:20 smithi159 podman[87667]: 2024-09-19 07:38:20.625811085 +0000 UTC m=+0.443215674 container start 56767aff7488a33c6c133a4cc4cef7c8d456ff5919faef29281c4f7955d62470 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:38:20.877 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:38:20 smithi159 podman[87667]: 2024-09-19 07:38:20.718418046 +0000 UTC m=+0.535822659 container attach 56767aff7488a33c6c133a4cc4cef7c8d456ff5919faef29281c4f7955d62470 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T07:38:21.137 INFO:journalctl@ceph.osd.0.smithi159.stdout:Sep 19 07:38:20 smithi159 podman[87667]: 2024-09-19 07:38:20.877658662 +0000 UTC m=+0.695063268 container died 56767aff7488a33c6c133a4cc4cef7c8d456ff5919faef29281c4f7955d62470 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T07:38:21.788 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T07:38:21.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:38:21.867 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T07:38:21.867 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T07:38:21.867 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.1 2024-09-19T07:38:22.215 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:38:21 smithi189 systemd[1]: Stopping Ceph osd.1 for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:38:22.556 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:38:22 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-1[39332]: 2024-09-19T07:38:22.214+0000 7f511ae60640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:38:22.556 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:38:22 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-1[39332]: 2024-09-19T07:38:22.214+0000 7f511ae60640 -1 osd.1 104 *** Got signal Terminated *** 2024-09-19T07:38:22.556 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:38:22 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-1[39332]: 2024-09-19T07:38:22.214+0000 7f511ae60640 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:38:27.806 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:38:27 smithi189 podman[67853]: 2024-09-19 07:38:27.543769398 +0000 UTC m=+5.474331512 container died 2fafa1e5599f46fa1a55170ca9295b72b3c4c67550a91974a537ab3e3e44a0c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T07:38:28.200 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:38:27 smithi189 podman[67853]: 2024-09-19 07:38:27.948886588 +0000 UTC m=+5.879448742 container cleanup 2fafa1e5599f46fa1a55170ca9295b72b3c4c67550a91974a537ab3e3e44a0c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T07:38:28.200 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:38:27 smithi189 bash[67853]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-1 2024-09-19T07:38:28.555 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:38:28 smithi189 podman[67869]: 2024-09-19 07:38:28.200287763 +0000 UTC m=+0.656246504 container remove 2fafa1e5599f46fa1a55170ca9295b72b3c4c67550a91974a537ab3e3e44a0c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-1, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T07:38:28.920 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:38:28 smithi189 podman[67956]: 2024-09-19 07:38:28.504590111 +0000 UTC m=+0.015996107 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:38:28.920 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:38:28 smithi189 podman[67956]: 2024-09-19 07:38:28.643068192 +0000 UTC m=+0.154474175 container create 225ccea352f7e6a25266085b1ea367c4b7cbf170aee236d94c75c0e320ab419c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T07:38:28.921 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:38:28 smithi189 podman[67956]: 2024-09-19 07:38:28.920902673 +0000 UTC m=+0.432308656 container init 225ccea352f7e6a25266085b1ea367c4b7cbf170aee236d94c75c0e320ab419c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T07:38:29.306 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:38:28 smithi189 podman[67956]: 2024-09-19 07:38:28.924165174 +0000 UTC m=+0.435571159 container start 225ccea352f7e6a25266085b1ea367c4b7cbf170aee236d94c75c0e320ab419c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-19T07:38:29.306 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:38:29 smithi189 podman[67956]: 2024-09-19 07:38:29.02096725 +0000 UTC m=+0.532373247 container attach 225ccea352f7e6a25266085b1ea367c4b7cbf170aee236d94c75c0e320ab419c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2) 2024-09-19T07:38:29.306 INFO:journalctl@ceph.osd.1.smithi189.stdout:Sep 19 07:38:29 smithi189 podman[67956]: 2024-09-19 07:38:29.112338451 +0000 UTC m=+0.623744437 container died 225ccea352f7e6a25266085b1ea367c4b7cbf170aee236d94c75c0e320ab419c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T07:38:29.789 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T07:38:29.829 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:38:29.829 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T07:38:29.829 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T07:38:29.830 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.2 2024-09-19T07:38:30.144 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:38:29 smithi203 systemd[1]: Stopping Ceph osd.2 for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:38:30.402 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:38:30 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-2[40318]: 2024-09-19T07:38:30.143+0000 7f9c6f349640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:38:30.402 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:38:30 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-2[40318]: 2024-09-19T07:38:30.143+0000 7f9c6f349640 -1 osd.2 104 *** Got signal Terminated *** 2024-09-19T07:38:30.402 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:38:30 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-2[40318]: 2024-09-19T07:38:30.143+0000 7f9c6f349640 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:38:35.972 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:38:35 smithi203 podman[69052]: 2024-09-19 07:38:35.677080589 +0000 UTC m=+5.656417562 container died cf6bc09e2c60c4e8435c2b7cdbfd71dda74ed714ba2a3a1988f897c07cb603d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T07:38:36.286 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:38:36 smithi203 podman[69052]: 2024-09-19 07:38:36.076850694 +0000 UTC m=+6.056187667 container cleanup cf6bc09e2c60c4e8435c2b7cdbfd71dda74ed714ba2a3a1988f897c07cb603d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T07:38:36.286 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:38:36 smithi203 bash[69052]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-2 2024-09-19T07:38:36.689 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:38:36 smithi203 podman[69080]: 2024-09-19 07:38:36.345096393 +0000 UTC m=+0.661847668 container remove cf6bc09e2c60c4e8435c2b7cdbfd71dda74ed714ba2a3a1988f897c07cb603d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T07:38:36.965 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:38:36 smithi203 podman[69169]: 2024-09-19 07:38:36.592999927 +0000 UTC m=+0.015916510 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:38:36.965 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:38:36 smithi203 podman[69169]: 2024-09-19 07:38:36.729474799 +0000 UTC m=+0.152391412 container create f6b8d9614efcf900dd2fe83de15ddf07b60f3f82c08050f552f096a2c037af7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731) 2024-09-19T07:38:37.286 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:38:36 smithi203 podman[69169]: 2024-09-19 07:38:36.965520309 +0000 UTC m=+0.388436876 container init f6b8d9614efcf900dd2fe83de15ddf07b60f3f82c08050f552f096a2c037af7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T07:38:37.286 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:38:36 smithi203 podman[69169]: 2024-09-19 07:38:36.968868953 +0000 UTC m=+0.391785534 container start f6b8d9614efcf900dd2fe83de15ddf07b60f3f82c08050f552f096a2c037af7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-19T07:38:37.286 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:38:37 smithi203 podman[69169]: 2024-09-19 07:38:37.040696307 +0000 UTC m=+0.463612893 container attach f6b8d9614efcf900dd2fe83de15ddf07b60f3f82c08050f552f096a2c037af7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-19T07:38:37.287 INFO:journalctl@ceph.osd.2.smithi203.stdout:Sep 19 07:38:37 smithi203 podman[69169]: 2024-09-19 07:38:37.157567946 +0000 UTC m=+0.580484532 container died f6b8d9614efcf900dd2fe83de15ddf07b60f3f82c08050f552f096a2c037af7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-2-deactivate, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T07:38:37.839 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T07:38:37.877 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:38:37.878 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T07:38:37.878 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T07:38:37.878 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.3 2024-09-19T07:38:38.296 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:38:37 smithi159 systemd[1]: Stopping Ceph osd.3 for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:38:38.637 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:38:38 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-3[51219]: 2024-09-19T07:38:38.294+0000 7f5917400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:38:38.637 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:38:38 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-3[51219]: 2024-09-19T07:38:38.294+0000 7f5917400640 -1 osd.3 104 *** Got signal Terminated *** 2024-09-19T07:38:38.637 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:38:38 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-3[51219]: 2024-09-19T07:38:38.294+0000 7f5917400640 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:38:39.786 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:39 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:39.345+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:40.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:40 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:40.054+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:40.786 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:40 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:40.319+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:40.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:40 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:40.585+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:41.288 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:40 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:40.846+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:41.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:41 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:41.014+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:41.786 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:41 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:41.293+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:41.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:41 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:41.605+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:42.156 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:41 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:41.868+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:42.305 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:42 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:42.011+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:42.386 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:38:42 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-3[51219]: 2024-09-19T07:38:42.063+0000 7f590de00640 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.350799+0000 front 2024-09-19T07:38:18.350768+0000 (oldest deadline 2024-09-19T07:38:41.850959+0000) 2024-09-19T07:38:42.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:42 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:42.155+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:42.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:42 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:42.287+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:42.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:42 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:42.574+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:43.145 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:42 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:42.875+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:43.305 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:43 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:43.023+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:43.387 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:38:43 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-3[51219]: 2024-09-19T07:38:43.093+0000 7f590de00640 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.350799+0000 front 2024-09-19T07:38:18.350768+0000 (oldest deadline 2024-09-19T07:38:41.850959+0000) 2024-09-19T07:38:43.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:43 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:43.144+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:43.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:43 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:43.265+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:43.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:43 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:43.620+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:43.887 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:38:43 smithi159 podman[87779]: 2024-09-19 07:38:43.743965055 +0000 UTC m=+5.575732247 container died d4826fbbdb5e441d8bc5a1a255175052cf53dd6e1bda8051c87a1934bf49d200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-09-19T07:38:44.133 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:43 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:43.838+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:44.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:44 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:44.026+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:44.387 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:38:44 smithi159 podman[87779]: 2024-09-19 07:38:44.283705872 +0000 UTC m=+6.115473050 container cleanup d4826fbbdb5e441d8bc5a1a255175052cf53dd6e1bda8051c87a1934bf49d200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:38:44.387 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:38:44 smithi159 bash[87779]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-3 2024-09-19T07:38:44.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:44 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:44.132+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:44.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:44 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:44.229+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:44.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:44 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:44.621+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:44.887 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:38:44 smithi159 podman[87798]: 2024-09-19 07:38:44.527081338 +0000 UTC m=+0.775313371 container remove d4826fbbdb5e441d8bc5a1a255175052cf53dd6e1bda8051c87a1934bf49d200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T07:38:45.106 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:44 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:44.823+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:45.305 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:45 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:45.024+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:45.387 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:38:45 smithi159 podman[87884]: 2024-09-19 07:38:44.998579284 +0000 UTC m=+0.027779068 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:38:45.387 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:38:45 smithi159 podman[87884]: 2024-09-19 07:38:45.127982903 +0000 UTC m=+0.157182685 container create a8be1ddd2bf1bfc131a14487af2275ff9817265bac5b27ea6a1a2629f7bf49d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-19T07:38:45.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:45 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:45.106+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:45.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:45 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:45.182+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:45.699 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:45 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:45.625+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:45.700 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:38:45 smithi159 podman[87884]: 2024-09-19 07:38:45.430867692 +0000 UTC m=+0.460067474 container init a8be1ddd2bf1bfc131a14487af2275ff9817265bac5b27ea6a1a2629f7bf49d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-19T07:38:45.700 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:38:45 smithi159 podman[87884]: 2024-09-19 07:38:45.43790505 +0000 UTC m=+0.467104833 container start a8be1ddd2bf1bfc131a14487af2275ff9817265bac5b27ea6a1a2629f7bf49d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T07:38:45.700 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:38:45 smithi159 podman[87884]: 2024-09-19 07:38:45.522066227 +0000 UTC m=+0.551266010 container attach a8be1ddd2bf1bfc131a14487af2275ff9817265bac5b27ea6a1a2629f7bf49d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:38:46.092 INFO:journalctl@ceph.osd.3.smithi159.stdout:Sep 19 07:38:45 smithi159 podman[87884]: 2024-09-19 07:38:45.700216535 +0000 UTC m=+0.729416319 container died a8be1ddd2bf1bfc131a14487af2275ff9817265bac5b27ea6a1a2629f7bf49d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T07:38:46.207 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:45 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:45.868+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:46.208 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:46 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:46.087+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:46.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:46 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:46.007+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:46.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:46 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:46.207+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:46.597 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T07:38:46.674 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:38:46.675 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T07:38:46.675 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T07:38:46.675 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.4 2024-09-19T07:38:46.886 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:46 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:46.609+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:46.995 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:38:46 smithi189 systemd[1]: Stopping Ceph osd.4 for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:38:47.125 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:46 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:46.820+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:47.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:47 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:47.016+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:47.306 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:38:46 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-4[45783]: 2024-09-19T07:38:46.995+0000 7f5063850640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:38:47.306 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:38:46 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-4[45783]: 2024-09-19T07:38:46.995+0000 7f5063850640 -1 osd.4 104 *** Got signal Terminated *** 2024-09-19T07:38:47.306 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:38:46 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-4[45783]: 2024-09-19T07:38:46.995+0000 7f5063850640 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:38:47.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:47 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:47.125+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:47.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:47 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:47.161+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:48.085 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:47 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:47.829+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:48.136 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:47 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:47.647+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:48.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:48 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:48.084+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:48.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:48 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:48.195+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:48.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:48 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:48.060+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:48.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:48 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:48.060+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:38:49.137 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:48 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:48.642+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:49.174 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:49 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:49.077+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:49.174 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:48 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:48.841+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:49.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:49 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:49.049+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:49.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:49 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:49.049+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:38:49.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:49 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:49.174+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:49.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:49 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:49.595+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:49.887 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:38:49 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:38:49.738+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:38:50.116 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:49 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:49.830+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:50.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:38:49 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:38:49.992+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:38:50.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:50 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:50.022+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:50.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:50 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:50.022+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:38:50.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:50 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:50.200+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:50.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:50 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:50.116+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:50.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:50 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:50.116+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.841958+0000 front 2024-09-19T07:38:23.841846+0000 (oldest deadline 2024-09-19T07:38:49.741900+0000) 2024-09-19T07:38:50.887 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:38:50 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:38:50.725+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:38:50.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:50 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:50.569+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:50.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:50 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:50.569+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:38:51.099 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:50 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:50.815+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:51.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:38:50 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:38:50.963+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:38:51.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:50 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:50.994+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:51.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:50 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:50.994+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:38:51.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:51 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:51.099+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:51.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:51 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:51.099+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.841958+0000 front 2024-09-19T07:38:23.841846+0000 (oldest deadline 2024-09-19T07:38:49.741900+0000) 2024-09-19T07:38:51.537 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:51 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:51.187+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:51.537 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:51 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:51.187+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:38:51.806 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:38:51 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-4[45783]: 2024-09-19T07:38:51.384+0000 7f505fe66640 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:26.645248+0000 front 2024-09-19T07:38:26.645288+0000 (oldest deadline 2024-09-19T07:38:51.345428+0000) 2024-09-19T07:38:51.886 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:38:51 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:38:51.753+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:38:51.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:51 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:51.533+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:51.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:51 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:51.533+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:38:52.064 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:51 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:51.805+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:52.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:38:51 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:38:51.928+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:38:52.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:51 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:51.987+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:52.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:51 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:51.987+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:38:52.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:52 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:52.064+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:52.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:52 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:52.064+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.841958+0000 front 2024-09-19T07:38:23.841846+0000 (oldest deadline 2024-09-19T07:38:49.741900+0000) 2024-09-19T07:38:52.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:52 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:52.188+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:52.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:52 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:52.188+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:38:52.771 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:38:52 smithi189 podman[68070]: 2024-09-19 07:38:52.48933396 +0000 UTC m=+5.616836522 container died 5507a3924330d137b1737ddc61947cc153d6b1350a762efdc293791b8004a514 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-4, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T07:38:52.887 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:38:52 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:38:52.720+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:38:52.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:52 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:52.536+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:52.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:52 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:52.536+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:38:53.055 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:38:52 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:38:52.977+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:38:53.056 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:38:52 smithi189 podman[68070]: 2024-09-19 07:38:52.87055482 +0000 UTC m=+5.998057377 container cleanup 5507a3924330d137b1737ddc61947cc153d6b1350a762efdc293791b8004a514 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T07:38:53.056 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:38:52 smithi189 bash[68070]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-4 2024-09-19T07:38:53.056 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:52 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:52.960+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:53.056 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:52 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:52.960+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:38:53.096 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:52 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:52.798+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:53.474 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:38:53 smithi189 podman[68085]: 2024-09-19 07:38:53.163725896 +0000 UTC m=+0.662966586 container remove 5507a3924330d137b1737ddc61947cc153d6b1350a762efdc293791b8004a514 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T07:38:53.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:53 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:53.096+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:53.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:53 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:53.096+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.841958+0000 front 2024-09-19T07:38:23.841846+0000 (oldest deadline 2024-09-19T07:38:49.741900+0000) 2024-09-19T07:38:53.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:53 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:53.230+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:53.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:53 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:53.230+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:38:53.791 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:38:53 smithi189 podman[68175]: 2024-09-19 07:38:53.377547797 +0000 UTC m=+0.015850824 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:38:53.792 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:38:53 smithi189 podman[68175]: 2024-09-19 07:38:53.506253982 +0000 UTC m=+0.144557063 container create ed00f0f6ad67859d07e31e5fbb412198d207d516c7c06e28146292d82a925676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T07:38:53.886 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:38:53 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:38:53.717+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:38:53.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:53 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:53.510+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:53.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:53 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:53.510+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:38:54.056 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:38:53 smithi189 podman[68175]: 2024-09-19 07:38:53.792308056 +0000 UTC m=+0.430611074 container init ed00f0f6ad67859d07e31e5fbb412198d207d516c7c06e28146292d82a925676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:38:54.056 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:38:53 smithi189 podman[68175]: 2024-09-19 07:38:53.795763927 +0000 UTC m=+0.434066948 container start ed00f0f6ad67859d07e31e5fbb412198d207d516c7c06e28146292d82a925676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T07:38:54.056 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:38:53 smithi189 podman[68175]: 2024-09-19 07:38:53.900712715 +0000 UTC m=+0.539015734 container attach ed00f0f6ad67859d07e31e5fbb412198d207d516c7c06e28146292d82a925676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T07:38:54.056 INFO:journalctl@ceph.osd.4.smithi189.stdout:Sep 19 07:38:53 smithi189 podman[68175]: 2024-09-19 07:38:53.982532546 +0000 UTC m=+0.620835567 container died ed00f0f6ad67859d07e31e5fbb412198d207d516c7c06e28146292d82a925676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T07:38:54.057 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:38:53 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:38:53.973+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:38:54.057 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:53 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:53.954+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:54.057 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:53 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:53.954+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:38:54.140 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:53 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:53.841+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:54.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:54 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:54.140+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:54.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:54 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:54.140+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.841958+0000 front 2024-09-19T07:38:23.841846+0000 (oldest deadline 2024-09-19T07:38:49.741900+0000) 2024-09-19T07:38:54.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:54 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:54.275+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:54.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:54 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:54.275+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:38:54.653 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T07:38:54.693 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:38:54.693 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T07:38:54.693 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T07:38:54.693 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.5 2024-09-19T07:38:54.887 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:38:54 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:38:54.763+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:38:54.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:54 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:54.547+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:54.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:54 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:54.547+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:38:55.013 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:54 smithi203 systemd[1]: Stopping Ceph osd.5 for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:38:55.014 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:54 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:54.831+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:55.237 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:55 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:55.013+0000 7fb714bcd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:38:55.237 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:55 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:55.013+0000 7fb714bcd640 -1 osd.5 104 *** Got signal Terminated *** 2024-09-19T07:38:55.238 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:55 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:55.013+0000 7fb714bcd640 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:38:55.238 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:55 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:55.159+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:55.238 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:55 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:55.159+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.841958+0000 front 2024-09-19T07:38:23.841846+0000 (oldest deadline 2024-09-19T07:38:49.741900+0000) 2024-09-19T07:38:55.238 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:55 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:55.236+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:55.238 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:55 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:55.236+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:38:55.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:38:54 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:38:54.984+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:38:55.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:54 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:54.912+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:55.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:54 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:54.912+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:38:55.809 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:38:55 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:38:55.746+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:38:55.809 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:55 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:55.514+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:55.809 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:55 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:55.514+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:38:56.259 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:55 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:55.876+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:56.259 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:56 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:56.116+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:56.259 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:56 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:56.116+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.841958+0000 front 2024-09-19T07:38:23.841846+0000 (oldest deadline 2024-09-19T07:38:49.741900+0000) 2024-09-19T07:38:56.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:38:56 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:38:56.027+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:38:56.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:38:56 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:38:56.027+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:38:56.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:55 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:55.956+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:56.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:55 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:55.956+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:38:56.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:56 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:56.259+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:56.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:56 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:56.259+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:38:56.785 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:56 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:56.510+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:56.786 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:56 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:56.510+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:38:57.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:38:56 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:38:56.785+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:38:57.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:38:56 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:38:56.785+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.540271+0000 front 2024-09-19T07:38:31.540205+0000 (oldest deadline 2024-09-19T07:38:56.240427+0000) 2024-09-19T07:38:57.153 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:56 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:56.891+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:57.207 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:38:57 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:38:57.006+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:38:57.207 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:38:57 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:38:57.006+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:38:57.207 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:56 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:56.934+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:57.207 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:56 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:56.934+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:38:57.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:57 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:57.153+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:57.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:57 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:57.153+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.841958+0000 front 2024-09-19T07:38:23.841846+0000 (oldest deadline 2024-09-19T07:38:49.741900+0000) 2024-09-19T07:38:57.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:57 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:57.153+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:32.543346+0000 front 2024-09-19T07:38:32.543246+0000 (oldest deadline 2024-09-19T07:38:56.643356+0000) 2024-09-19T07:38:57.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:57 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:57.245+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:57.537 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:57 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:57.245+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:38:57.798 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:57 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:57.521+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:57.799 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:57 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:57.521+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:38:57.799 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:57 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:57.521+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:38:58.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:38:57 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:38:57.798+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:38:58.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:38:57 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:38:57.798+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.540271+0000 front 2024-09-19T07:38:31.540205+0000 (oldest deadline 2024-09-19T07:38:56.240427+0000) 2024-09-19T07:38:58.223 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:38:57 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:38:57.971+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:38:58.224 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:38:57 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:38:57.971+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:38:58.224 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:57 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:57.948+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:58.224 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:57 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:57.948+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:38:58.225 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:57 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:57.948+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:38:58.263 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:58 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:58.120+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:58.263 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:58 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:58.120+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.841958+0000 front 2024-09-19T07:38:23.841846+0000 (oldest deadline 2024-09-19T07:38:49.741900+0000) 2024-09-19T07:38:58.263 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:58 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:58.120+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:32.543346+0000 front 2024-09-19T07:38:32.543246+0000 (oldest deadline 2024-09-19T07:38:56.643356+0000) 2024-09-19T07:38:58.263 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:57 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:57.875+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:58.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:58 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:58.263+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:58.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:58 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:58.263+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:38:58.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:58 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:58.263+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:38:58.887 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:38:58 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:38:58.769+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:38:58.887 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:38:58 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:38:58.769+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.540271+0000 front 2024-09-19T07:38:31.540205+0000 (oldest deadline 2024-09-19T07:38:56.240427+0000) 2024-09-19T07:38:58.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:58 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:58.527+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:58.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:58 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:58.527+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:38:58.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:58 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:58.527+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:38:59.143 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:58 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:58.858+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:38:59.144 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:58 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:58.858+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:38:59.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:38:58 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:38:58.968+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:38:59.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:38:58 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:38:58.968+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:38:59.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:58 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:58.966+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:38:59.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:58 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:58.966+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:38:59.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:58 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:58.966+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:38:59.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:59 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:59.143+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.140180+0000 front 2024-09-19T07:38:18.140194+0000 (oldest deadline 2024-09-19T07:38:41.640293+0000) 2024-09-19T07:38:59.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:59 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:59.143+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.841958+0000 front 2024-09-19T07:38:23.841846+0000 (oldest deadline 2024-09-19T07:38:49.741900+0000) 2024-09-19T07:38:59.536 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:38:59 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5[47221]: 2024-09-19T07:38:59.143+0000 7fb7111e3640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:32.543346+0000 front 2024-09-19T07:38:32.543246+0000 (oldest deadline 2024-09-19T07:38:56.643356+0000) 2024-09-19T07:38:59.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:59 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:59.246+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:38:59.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:59 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:59.246+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:38:59.537 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:38:59 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:38:59.246+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:38:59.762 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:59 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:59.483+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:38:59.762 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:59 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:59.483+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:38:59.762 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:38:59 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:38:59.483+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:00.136 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:38:59 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:38:59.762+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:39:00.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:38:59 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:38:59.762+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.540271+0000 front 2024-09-19T07:38:31.540205+0000 (oldest deadline 2024-09-19T07:38:56.240427+0000) 2024-09-19T07:39:00.198 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:59 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:59.890+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:00.198 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:38:59 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:38:59.890+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:00.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:00 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:00.002+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:39:00.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:00 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:00.002+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:39:00.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:59 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:59.919+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:00.307 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:59 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:59.919+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:00.307 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:38:59 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:38:59.919+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:00.473 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:39:00 smithi203 podman[69271]: 2024-09-19 07:39:00.427320723 +0000 UTC m=+5.535584860 container died 03ada340798bd9974b50557b21a34241341ec1ee552328b100f319e675ee869e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T07:39:00.474 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:00 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:00.198+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:00.474 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:00 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:00.198+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:00.474 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:00 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:00.198+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:00.807 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:00 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:00.436+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:00.807 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:00 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:00.436+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:00.807 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:00 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:00.436+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:01.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:00 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:00.868+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:01.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:00 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:00.868+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:01.036 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:39:00 smithi203 podman[69271]: 2024-09-19 07:39:00.880505676 +0000 UTC m=+5.988769819 container cleanup 03ada340798bd9974b50557b21a34241341ec1ee552328b100f319e675ee869e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:39:01.037 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:39:00 smithi203 bash[69271]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5 2024-09-19T07:39:01.136 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:00 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:00.807+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:39:01.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:00 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:00.807+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.540271+0000 front 2024-09-19T07:38:31.540205+0000 (oldest deadline 2024-09-19T07:38:56.240427+0000) 2024-09-19T07:39:01.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:01 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:01.051+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:39:01.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:01 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:01.051+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:39:01.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:00 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:00.916+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:01.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:00 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:00.916+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:01.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:00 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:00.916+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:01.475 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:39:01 smithi203 podman[69290]: 2024-09-19 07:39:01.190393818 +0000 UTC m=+0.760356701 container remove 03ada340798bd9974b50557b21a34241341ec1ee552328b100f319e675ee869e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True) 2024-09-19T07:39:01.476 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:01 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:01.160+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:01.476 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:01 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:01.160+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:01.476 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:01 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:01.160+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:01.786 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:39:01 smithi203 podman[69388]: 2024-09-19 07:39:01.449133781 +0000 UTC m=+0.016154746 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:39:01.786 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:39:01 smithi203 podman[69388]: 2024-09-19 07:39:01.574703435 +0000 UTC m=+0.141724391 container create 46ee6d2ec42043ae474dde1a234dd8a00a8d20ca50a806c513f9b10f209cfded (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-19T07:39:01.792 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:01 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:01.438+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:01.792 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:01 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:01.438+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:01.792 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:01 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:01.438+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:02.127 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:01 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:01.904+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:02.127 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:01 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:01.904+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:02.127 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:39:01 smithi203 podman[69388]: 2024-09-19 07:39:01.835860456 +0000 UTC m=+0.402881421 container init 46ee6d2ec42043ae474dde1a234dd8a00a8d20ca50a806c513f9b10f209cfded (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T07:39:02.128 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:39:01 smithi203 podman[69388]: 2024-09-19 07:39:01.839097968 +0000 UTC m=+0.406118929 container start 46ee6d2ec42043ae474dde1a234dd8a00a8d20ca50a806c513f9b10f209cfded (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T07:39:02.128 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:39:01 smithi203 podman[69388]: 2024-09-19 07:39:01.910887623 +0000 UTC m=+0.477908599 container attach 46ee6d2ec42043ae474dde1a234dd8a00a8d20ca50a806c513f9b10f209cfded (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2) 2024-09-19T07:39:02.128 INFO:journalctl@ceph.osd.5.smithi203.stdout:Sep 19 07:39:02 smithi203 podman[69388]: 2024-09-19 07:39:02.037507227 +0000 UTC m=+0.604528190 container died 46ee6d2ec42043ae474dde1a234dd8a00a8d20ca50a806c513f9b10f209cfded (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux ) 2024-09-19T07:39:02.136 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:01 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:01.791+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:39:02.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:01 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:01.791+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.540271+0000 front 2024-09-19T07:38:31.540205+0000 (oldest deadline 2024-09-19T07:38:56.240427+0000) 2024-09-19T07:39:02.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:02 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:02.001+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:39:02.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:02 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:02.001+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:39:02.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:01 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:01.940+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:02.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:01 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:01.940+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:02.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:01 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:01.940+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:02.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:02 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:02.127+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:02.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:02 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:02.127+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:02.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:02 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:02.127+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:02.696 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T07:39:02.737 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:39:02.737 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T07:39:02.737 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T07:39:02.737 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.6 2024-09-19T07:39:02.749 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:02 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:02.433+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:02.749 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:02 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:02.433+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:02.749 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:02 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:02.433+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:03.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:02 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:02.748+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:39:03.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:02 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:02.748+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.540271+0000 front 2024-09-19T07:38:31.540205+0000 (oldest deadline 2024-09-19T07:38:56.240427+0000) 2024-09-19T07:39:03.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:02 smithi159 systemd[1]: Stopping Ceph osd.6 for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:39:03.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:02 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:02.954+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:03.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:02 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:02.954+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:03.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:03 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:03.156+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:03.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:03 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:03.156+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:03.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:03 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:03.156+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:03.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:03 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:03.046+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:39:03.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:03 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:03.046+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:39:03.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:02 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:02.927+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:03.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:02 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:02.927+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:03.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:02 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:02.927+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:03.637 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:03 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:03.164+0000 7f44e0800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:39:03.637 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:03 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:03.164+0000 7f44e0800640 -1 osd.6 104 *** Got signal Terminated *** 2024-09-19T07:39:03.637 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:03 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:03.164+0000 7f44e0800640 -1 osd.6 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:39:03.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:03 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:03.394+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:03.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:03 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:03.394+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:03.638 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:03 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:03.394+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:04.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:03 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:03.718+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:39:04.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:03 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:03.718+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.540271+0000 front 2024-09-19T07:38:31.540205+0000 (oldest deadline 2024-09-19T07:38:56.240427+0000) 2024-09-19T07:39:04.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:03 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:03.991+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:04.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:03 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:03.991+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:04.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:04 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:04.112+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:04.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:04 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:04.112+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:04.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:04 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:04.112+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:04.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:04 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:04.061+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:39:04.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:04 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:04.061+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:39:04.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:03 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:03.919+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:04.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:03 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:03.919+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:04.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:03 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:03.919+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:04.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:04 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:04.350+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:04.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:04 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:04.350+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:04.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:04 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:04.350+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:05.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:04 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:04.747+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:39:05.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:04 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:04.747+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.540271+0000 front 2024-09-19T07:38:31.540205+0000 (oldest deadline 2024-09-19T07:38:56.240427+0000) 2024-09-19T07:39:05.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:04 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:04.747+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:42.443815+0000 front 2024-09-19T07:38:42.443795+0000 (oldest deadline 2024-09-19T07:39:04.144029+0000) 2024-09-19T07:39:05.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:04 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:04.955+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:05.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:04 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:04.955+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:05.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:05 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:05.089+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:05.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:05 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:05.089+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:05.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:05 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:05.089+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:05.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:05 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:05.096+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:39:05.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:05 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:05.096+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:39:05.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:04 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:04.894+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:05.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:04 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:04.894+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:05.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:04 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:04.894+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:05.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:05.316+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:05.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:05.316+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:05.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:05.316+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:06.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:05.729+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:39:06.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:05.729+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.540271+0000 front 2024-09-19T07:38:31.540205+0000 (oldest deadline 2024-09-19T07:38:56.240427+0000) 2024-09-19T07:39:06.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:05 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:05.729+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:42.443815+0000 front 2024-09-19T07:38:42.443795+0000 (oldest deadline 2024-09-19T07:39:04.144029+0000) 2024-09-19T07:39:06.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:06 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:06.002+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:06.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:06 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:06.002+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:06.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:06 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:06.002+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:06.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:06 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:06.118+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:06.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:06 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:06.118+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:06.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:06 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:06.118+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:06.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:06 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:06.118+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:06.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:06 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:06.086+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:39:06.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:06 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:06.086+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:39:06.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:06 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:06.086+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:41.031804+0000 front 2024-09-19T07:38:41.031783+0000 (oldest deadline 2024-09-19T07:39:05.131709+0000) 2024-09-19T07:39:06.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:05 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:05.846+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:06.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:05 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:05.846+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:06.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:05 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:05.846+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:06.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:06 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:06.316+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:06.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:06 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:06.316+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:06.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:06 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:06.316+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:06.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:06 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:06.316+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:07.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:06 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:06.741+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:39:07.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:06 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:06.741+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.540271+0000 front 2024-09-19T07:38:31.540205+0000 (oldest deadline 2024-09-19T07:38:56.240427+0000) 2024-09-19T07:39:07.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:06 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:06.741+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:42.443815+0000 front 2024-09-19T07:38:42.443795+0000 (oldest deadline 2024-09-19T07:39:04.144029+0000) 2024-09-19T07:39:07.288 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:06 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:06.967+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:07.288 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:06 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:06.967+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:07.288 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:06 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:06.967+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:07.289 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:07 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:07.139+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:07.289 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:07 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:07.139+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:07.289 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:07 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:07.139+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:07.289 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:07 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:07.139+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:07.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:07 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:07.077+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:39:07.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:07 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:07.077+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:39:07.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:07 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:07.077+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:41.031804+0000 front 2024-09-19T07:38:41.031783+0000 (oldest deadline 2024-09-19T07:39:05.131709+0000) 2024-09-19T07:39:07.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:06 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:06.874+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:07.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:06 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:06.874+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:07.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:06 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:06.874+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:07.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:07 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:07.332+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:07.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:07 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:07.332+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:07.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:07 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:07.332+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:07.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:07 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:07.332+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:08.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:07 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:07.717+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.938778+0000 front 2024-09-19T07:38:23.938971+0000 (oldest deadline 2024-09-19T07:38:49.238967+0000) 2024-09-19T07:39:08.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:07 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:07.717+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.540271+0000 front 2024-09-19T07:38:31.540205+0000 (oldest deadline 2024-09-19T07:38:56.240427+0000) 2024-09-19T07:39:08.137 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:07 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6[58187]: 2024-09-19T07:39:07.717+0000 7f44dc200640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:42.443815+0000 front 2024-09-19T07:38:42.443795+0000 (oldest deadline 2024-09-19T07:39:04.144029+0000) 2024-09-19T07:39:08.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:07 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:07.954+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:08.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:07 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:07.954+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:08.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:07 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:07.954+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:08.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:08 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:08.178+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:08.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:08 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:08.178+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:08.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:08 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:08.178+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:08.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:08 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:08.178+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:08.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:08 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:08.034+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:39:08.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:08 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:08.034+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:39:08.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:08 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:08.034+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:41.031804+0000 front 2024-09-19T07:38:41.031783+0000 (oldest deadline 2024-09-19T07:39:05.131709+0000) 2024-09-19T07:39:08.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:07 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:07.874+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:08.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:07 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:07.874+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:08.307 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:07 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:07.874+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:08.307 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:07 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:07.874+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:08.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:08.334+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:08.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:08.334+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:08.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:08.334+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:08.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:08 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:08.334+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:09.014 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:08 smithi159 podman[87998]: 2024-09-19 07:39:08.697096456 +0000 UTC m=+5.676836671 container died e4d3c78cff897b96ee028410fcbae93fba4e81e84886783e5752e5ba75118e25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T07:39:09.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:09 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:09.001+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:09.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:09 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:09.001+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:09.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:09 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:09.001+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:09.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:09 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:09.162+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:09.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:09 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:09.162+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:09.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:09 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:09.162+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:09.288 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:09 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:09.162+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:09.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:09 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:09.004+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:39:09.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:09 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:09.004+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:39:09.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:09 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:09.004+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:41.031804+0000 front 2024-09-19T07:38:41.031783+0000 (oldest deadline 2024-09-19T07:39:05.131709+0000) 2024-09-19T07:39:09.307 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:08 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:08.922+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:09.307 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:08 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:08.923+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:09.307 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:08 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:08.923+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:09.307 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:08 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:08.923+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:09.324 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:09 smithi159 podman[87998]: 2024-09-19 07:39:09.161808429 +0000 UTC m=+6.141548637 container cleanup e4d3c78cff897b96ee028410fcbae93fba4e81e84886783e5752e5ba75118e25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T07:39:09.325 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:09 smithi159 bash[87998]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6 2024-09-19T07:39:09.636 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:09 smithi159 podman[88015]: 2024-09-19 07:39:09.330375431 +0000 UTC m=+0.626473963 container remove e4d3c78cff897b96ee028410fcbae93fba4e81e84886783e5752e5ba75118e25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-19T07:39:09.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:09 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:09.324+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:09.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:09 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:09.324+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:09.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:09 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:09.324+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:09.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:09 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:09.324+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:10.123 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:09 smithi159 podman[88101]: 2024-09-19 07:39:09.768601056 +0000 UTC m=+0.026922137 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:39:10.123 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:09 smithi159 podman[88101]: 2024-09-19 07:39:09.897897208 +0000 UTC m=+0.156218289 container create 6a18c31764d860840bad76d92b05910ca387d655123f254a28906b9a07fead6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T07:39:10.123 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:10 smithi159 podman[88101]: 2024-09-19 07:39:10.123428107 +0000 UTC m=+0.381749188 container init 6a18c31764d860840bad76d92b05910ca387d655123f254a28906b9a07fead6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-19T07:39:10.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:09 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:09.980+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:10.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:09 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:09.980+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:10.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:09 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:09.980+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:10.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:10 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:10.182+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:10.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:10 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:10.182+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:10.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:10 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:10.182+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:10.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:10 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:10.182+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:10.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:10 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:10.054+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:39:10.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:10 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:10.054+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:39:10.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:10 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:10.054+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:41.031804+0000 front 2024-09-19T07:38:41.031783+0000 (oldest deadline 2024-09-19T07:39:05.131709+0000) 2024-09-19T07:39:10.307 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:09 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:09.919+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:10.307 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:09 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:09.919+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:10.307 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:09 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:09.919+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:10.307 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:09 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:09.919+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:10.387 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:10 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:10.298+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:10.387 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:10 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:10.298+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:10.387 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:10 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:10.298+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:10.387 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:10 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:10.298+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:10.387 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:10 smithi159 podman[88101]: 2024-09-19 07:39:10.130404146 +0000 UTC m=+0.388725231 container start 6a18c31764d860840bad76d92b05910ca387d655123f254a28906b9a07fead6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T07:39:10.388 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:10 smithi159 podman[88101]: 2024-09-19 07:39:10.225342288 +0000 UTC m=+0.483663382 container attach 6a18c31764d860840bad76d92b05910ca387d655123f254a28906b9a07fead6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T07:39:10.647 INFO:journalctl@ceph.osd.6.smithi159.stdout:Sep 19 07:39:10 smithi159 podman[88125]: 2024-09-19 07:39:10.437984942 +0000 UTC m=+0.031315747 container died 6a18c31764d860840bad76d92b05910ca387d655123f254a28906b9a07fead6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T07:39:11.113 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T07:39:11.231 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:39:11.232 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T07:39:11.232 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T07:39:11.232 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.7 2024-09-19T07:39:11.261 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:11 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:11.050+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:39:11.261 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:11 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:11.050+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:39:11.261 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:11 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:11.050+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:41.031804+0000 front 2024-09-19T07:38:41.031783+0000 (oldest deadline 2024-09-19T07:39:05.131709+0000) 2024-09-19T07:39:11.261 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:10 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:10.870+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:11.261 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:10 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:10.870+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:11.261 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:10 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:10.870+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:11.262 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:10 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:10.870+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:11.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:11 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:11.030+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:11.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:11 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:11.030+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:11.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:11 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:11.030+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:11.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:11 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:11.198+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:11.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:11 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:11.198+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:11.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:11 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:11.198+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:11.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:11 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:11.198+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:11.555 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:11 smithi189 systemd[1]: Stopping Ceph osd.7 for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:39:11.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:11 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:11.274+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:11.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:11 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:11.274+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:11.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:11 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:11.274+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:11.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:11 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:11.274+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:11.911 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:11 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:11.557+0000 7efc96c0c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:39:11.912 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:11 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:11.557+0000 7efc96c0c640 -1 osd.7 104 *** Got signal Terminated *** 2024-09-19T07:39:11.912 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:11 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:11.557+0000 7efc96c0c640 -1 osd.7 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:39:12.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:12 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:12.005+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:12.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:12 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:12.005+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:12.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:12 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:12.005+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:12.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:12 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:12.214+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:12.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:12 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:12.214+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:12.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:12 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:12.214+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:12.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:12 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:12.214+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:12.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:12 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:12.066+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:39:12.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:12 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:12.066+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:39:12.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:12 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:12.066+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:41.031804+0000 front 2024-09-19T07:38:41.031783+0000 (oldest deadline 2024-09-19T07:39:05.131709+0000) 2024-09-19T07:39:12.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:11 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:11.911+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:12.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:11 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:11.911+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:12.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:11 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:11.911+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:12.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:11 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:11.911+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:12.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:12 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:12.302+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:12.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:12 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:12.302+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:12.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:12 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:12.302+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:12.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:12 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:12.302+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:13.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:13 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:13.093+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:39:13.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:13 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:13.093+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:39:13.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:13 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:13.093+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:41.031804+0000 front 2024-09-19T07:38:41.031783+0000 (oldest deadline 2024-09-19T07:39:05.131709+0000) 2024-09-19T07:39:13.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:13 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:13.093+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:48.433506+0000 front 2024-09-19T07:38:48.433532+0000 (oldest deadline 2024-09-19T07:39:12.533803+0000) 2024-09-19T07:39:13.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:12 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:12.909+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:13.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:12 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:12.909+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:13.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:12 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:12.909+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:13.307 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:12 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:12.909+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:13.536 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:13 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:13.055+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:13.536 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:13 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:13.055+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:13.536 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:13 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:13.055+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:13.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:13 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:13.257+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:13.537 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:13 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:13.257+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:13.537 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:13 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:13.257+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:13.537 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:13 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:13.257+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:13.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:13 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:13.295+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:13.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:13 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:13.295+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:13.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:13 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:13.295+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:13.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:13 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:13.295+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:14.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:14 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:14.007+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:14.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:14 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:14.007+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:14.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:14 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:14.007+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:14.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:14 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:14.242+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:14.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:14 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:14.242+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:14.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:14 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:14.242+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:14.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:14 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:14.242+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:14.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:14 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:14.073+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:39:14.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:14 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:14.073+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:39:14.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:14 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:14.073+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:41.031804+0000 front 2024-09-19T07:38:41.031783+0000 (oldest deadline 2024-09-19T07:39:05.131709+0000) 2024-09-19T07:39:14.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:14 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:14.073+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:48.433506+0000 front 2024-09-19T07:38:48.433532+0000 (oldest deadline 2024-09-19T07:39:12.533803+0000) 2024-09-19T07:39:14.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:13 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:13.940+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:14.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:13 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:13.940+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:14.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:13 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:13.940+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:14.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:13 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:13.940+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:14.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:14 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:14.264+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:14.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:14 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:14.264+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:14.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:14 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:14.264+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:14.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:14 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:14.264+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:15.281 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:14 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:14.975+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:15.281 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:14 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:14.975+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:15.281 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:14 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:14.975+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:15.281 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:14 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:14.975+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:15.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:14 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:14.977+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:15.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:14 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:14.977+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:15.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:14 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:14.977+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:15.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:14 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:14.977+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:15.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:15 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:15.122+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:39:15.307 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:15 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:15.122+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:39:15.307 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:15 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:15.122+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:41.031804+0000 front 2024-09-19T07:38:41.031783+0000 (oldest deadline 2024-09-19T07:39:05.131709+0000) 2024-09-19T07:39:15.307 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:15 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:15.122+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:48.433506+0000 front 2024-09-19T07:38:48.433532+0000 (oldest deadline 2024-09-19T07:39:12.533803+0000) 2024-09-19T07:39:15.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:15 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:15.281+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:15.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:15 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:15.281+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:15.537 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:15 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:15.281+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:15.537 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:15 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:15.281+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:15.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:15 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:15.257+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:15.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:15 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:15.257+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:15.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:15 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:15.257+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:15.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:15 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:15.257+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:16.237 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:15 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:15.967+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:16.238 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:15 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:15.967+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:16.238 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:15 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:15.967+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:16.238 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:15 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:15.967+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:16.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:15 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:15.946+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:16.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:15 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:15.946+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:16.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:15 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:15.946+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:16.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:15 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:15.946+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:16.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:16 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:16.141+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:25.130177+0000 front 2024-09-19T07:38:25.130199+0000 (oldest deadline 2024-09-19T07:38:49.830308+0000) 2024-09-19T07:39:16.306 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:16 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:16.141+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:29.830740+0000 front 2024-09-19T07:38:29.830867+0000 (oldest deadline 2024-09-19T07:38:55.730772+0000) 2024-09-19T07:39:16.307 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:16 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:16.141+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:41.031804+0000 front 2024-09-19T07:38:41.031783+0000 (oldest deadline 2024-09-19T07:39:05.131709+0000) 2024-09-19T07:39:16.307 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:16 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7[52544]: 2024-09-19T07:39:16.141+0000 7efc93222640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:48.433506+0000 front 2024-09-19T07:38:48.433532+0000 (oldest deadline 2024-09-19T07:39:12.533803+0000) 2024-09-19T07:39:16.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:16 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:16.237+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:16.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:16 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:16.237+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:16.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:16 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:16.237+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:16.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:16 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:16.237+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:16.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:16 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:16.305+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:16.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:16 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:16.305+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:16.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:16 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:16.305+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:16.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:16 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:16.305+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:17.250 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:16 smithi189 podman[68289]: 2024-09-19 07:39:16.976335078 +0000 UTC m=+5.542391256 container died 942984d78e4cea86e297f64c14da3d558543929d681c136e8e3806efbb210b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4) 2024-09-19T07:39:17.250 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:16 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:16.912+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:17.250 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:16 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:16.912+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:17.250 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:16 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:16.912+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:17.251 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:16 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:16.912+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:17.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:17 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:17.203+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:17.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:17 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:17.203+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:17.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:17 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:17.203+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:17.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:17 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:17.203+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:17.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:17 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:17.017+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:17.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:17 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:17.017+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:17.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:17 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:17.017+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:17.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:17 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:17.017+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:17.556 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:17 smithi189 podman[68289]: 2024-09-19 07:39:17.357397213 +0000 UTC m=+5.923453381 container cleanup 942984d78e4cea86e297f64c14da3d558543929d681c136e8e3806efbb210b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T07:39:17.556 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:17 smithi189 bash[68289]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7 2024-09-19T07:39:17.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:17 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:17.336+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:17.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:17 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:17.336+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:17.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:17 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:17.336+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:17.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:17 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:17.336+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:17.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:17 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:17.336+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.769717+0000 front 2024-09-19T07:38:51.769971+0000 (oldest deadline 2024-09-19T07:39:16.469248+0000) 2024-09-19T07:39:17.931 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:17 smithi189 podman[68307]: 2024-09-19 07:39:17.609110822 +0000 UTC m=+0.627582106 container remove 942984d78e4cea86e297f64c14da3d558543929d681c136e8e3806efbb210b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-19T07:39:18.221 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:17 smithi189 podman[68392]: 2024-09-19 07:39:17.834636366 +0000 UTC m=+0.015844587 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:39:18.221 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:17 smithi189 podman[68392]: 2024-09-19 07:39:17.959884792 +0000 UTC m=+0.141092999 container create c7717450bcebc6e78dfaf186674bd97978b526d98e7c7d7852d2247c0e3faa77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T07:39:18.221 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:18 smithi189 podman[68392]: 2024-09-19 07:39:18.220960417 +0000 UTC m=+0.402168626 container init c7717450bcebc6e78dfaf186674bd97978b526d98e7c7d7852d2247c0e3faa77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T07:39:18.221 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:17 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:17.930+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:18.221 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:17 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:17.930+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:18.221 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:17 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:17.930+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:18.221 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:17 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:17.930+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:18.222 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:17 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:17.930+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:18.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:18 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:18.157+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:18.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:18 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:18.158+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:18.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:18 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:18.158+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:18.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:18 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:18.158+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:18.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:18 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:18.012+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:18.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:18 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:18.012+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:18.288 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:18 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:18.012+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:18.288 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:18 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:18.012+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:18.556 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:18 smithi189 podman[68392]: 2024-09-19 07:39:18.224368783 +0000 UTC m=+0.405576991 container start c7717450bcebc6e78dfaf186674bd97978b526d98e7c7d7852d2247c0e3faa77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=) 2024-09-19T07:39:18.556 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:18 smithi189 podman[68392]: 2024-09-19 07:39:18.312727996 +0000 UTC m=+0.493936204 container attach c7717450bcebc6e78dfaf186674bd97978b526d98e7c7d7852d2247c0e3faa77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True) 2024-09-19T07:39:18.556 INFO:journalctl@ceph.osd.7.smithi189.stdout:Sep 19 07:39:18 smithi189 podman[68392]: 2024-09-19 07:39:18.418294118 +0000 UTC m=+0.599502338 container died c7717450bcebc6e78dfaf186674bd97978b526d98e7c7d7852d2247c0e3faa77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:39:18.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:18 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:18.339+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:18.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:18 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:18.340+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:18.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:18 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:18.340+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:18.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:18 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:18.340+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:18.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:18 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:18.340+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.769717+0000 front 2024-09-19T07:38:51.769971+0000 (oldest deadline 2024-09-19T07:39:16.469248+0000) 2024-09-19T07:39:19.041 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:18 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:18.945+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:19.041 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:18 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:18.945+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:19.041 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:18 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:18.945+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:19.041 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:18 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:18.945+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:19.042 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:18 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:18.945+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:19.066 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T07:39:19.146 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:39:19.147 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T07:39:19.147 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-19T07:39:19.147 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.8 2024-09-19T07:39:19.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:19 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:19.016+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:19.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:19 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:19.016+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:19.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:19 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:19.016+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:19.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:19 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:19.016+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:19.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:19 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:19.197+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:19.288 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:19 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:19.197+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:19.288 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:19 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:19.197+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:19.288 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:19 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:19.197+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:19.288 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:19 smithi203 systemd[1]: Stopping Ceph osd.8 for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:39:19.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:19 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:19.293+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:19.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:19 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:19.293+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:19.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:19 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:19.293+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:19.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:19 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:19.293+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:19.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:19 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:19.293+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.769717+0000 front 2024-09-19T07:38:51.769971+0000 (oldest deadline 2024-09-19T07:39:16.469248+0000) 2024-09-19T07:39:19.786 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:19 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:19.465+0000 7fdddeb1e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:39:19.786 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:19 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:19.465+0000 7fdddeb1e640 -1 osd.8 104 *** Got signal Terminated *** 2024-09-19T07:39:19.786 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:19 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:19.465+0000 7fdddeb1e640 -1 osd.8 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:39:20.277 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:19 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:19.935+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:20.277 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:19 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:19.935+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:20.278 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:19 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:19.935+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:20.278 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:19 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:19.935+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:20.278 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:19 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:19.935+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:20.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:20 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:20.149+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:20.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:20 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:20.149+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:20.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:20 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:20.149+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:20.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:20 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:20.149+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:20.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:20 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:20.013+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:20.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:20 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:20.013+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:20.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:20 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:20.013+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:20.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:20 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:20.013+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:20.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:20 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:20.256+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:20.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:20 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:20.256+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:20.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:20 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:20.256+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:20.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:20 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:20.256+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:20.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:20 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:20.256+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.769717+0000 front 2024-09-19T07:38:51.769971+0000 (oldest deadline 2024-09-19T07:39:16.469248+0000) 2024-09-19T07:39:21.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:20 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:20.979+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:21.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:20 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:20.979+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:21.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:20 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:20.979+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:21.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:20 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:20.979+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:21.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:20 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:20.979+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:21.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:20 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:20.979+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:21.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:21 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:21.187+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:21.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:21 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:21.187+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:21.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:21 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:21.187+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:21.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:21 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:21.187+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:21.536 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:21 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:21.051+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:21.537 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:21 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:21.051+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:21.537 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:21 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:21.051+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:21.537 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:21 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:21.051+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:21.585 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:21 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:21.259+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:21.585 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:21 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:21.259+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:21.585 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:21 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:21.259+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:21.585 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:21 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:21.259+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:21.585 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:21 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:21.259+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.769717+0000 front 2024-09-19T07:38:51.769971+0000 (oldest deadline 2024-09-19T07:39:16.469248+0000) 2024-09-19T07:39:22.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:22 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:22.013+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:22.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:22 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:22.013+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:22.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:22 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:22.013+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:22.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:22 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:22.013+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:22.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:22 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:22.013+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:22.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:22 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:22.013+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:22.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:22 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:22.163+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:22.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:22 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:22.163+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:22.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:22 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:22.163+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:22.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:22 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:22.163+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:22.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:22 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:22.163+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:58.613701+0000 front 2024-09-19T07:38:58.613653+0000 (oldest deadline 2024-09-19T07:39:22.113805+0000) 2024-09-19T07:39:22.537 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:22 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:22.039+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:22.537 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:22 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:22.039+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:22.537 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:22 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:22.039+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:22.537 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:22 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:22.039+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:22.537 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:22 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:22.039+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:22.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:22 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:22.210+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:22.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:22 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:22.210+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:22.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:22 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:22.210+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:22.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:22 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:22.210+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:22.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:22 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:22.210+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.769717+0000 front 2024-09-19T07:38:51.769971+0000 (oldest deadline 2024-09-19T07:39:16.469248+0000) 2024-09-19T07:39:23.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:23 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:23.114+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:23.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:23 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:23.114+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:23.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:23 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:23.114+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:23.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:23 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:23.114+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:23.287 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:23 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:23.114+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:58.613701+0000 front 2024-09-19T07:38:58.613653+0000 (oldest deadline 2024-09-19T07:39:22.113805+0000) 2024-09-19T07:39:23.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:23 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:23.018+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:23.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:23 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:23.018+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:23.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:23 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:23.018+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:23.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:23 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:23.018+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:23.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:23 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:23.018+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:23.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:23 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:23.037+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:23.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:23 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:23.037+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:23.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:23 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:23.037+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:23.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:23 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:23.037+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:23.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:23 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:23.037+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:23.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:23 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:23.037+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:23.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:23 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:23.216+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:23.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:23 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:23.216+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:23.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:23 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:23.216+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:23.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:23 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:23.216+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:23.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:23 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:23.216+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.769717+0000 front 2024-09-19T07:38:51.769971+0000 (oldest deadline 2024-09-19T07:39:16.469248+0000) 2024-09-19T07:39:24.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:24 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:24.077+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.807813+0000 front 2024-09-19T07:38:14.807873+0000 (oldest deadline 2024-09-19T07:38:38.907614+0000) 2024-09-19T07:39:24.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:24 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:24.077+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:27.009934+0000 front 2024-09-19T07:38:27.009846+0000 (oldest deadline 2024-09-19T07:38:51.109889+0000) 2024-09-19T07:39:24.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:24 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:24.077+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:31.110256+0000 front 2024-09-19T07:38:31.110166+0000 (oldest deadline 2024-09-19T07:38:57.510760+0000) 2024-09-19T07:39:24.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:24 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:24.077+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:39.211993+0000 front 2024-09-19T07:38:39.212050+0000 (oldest deadline 2024-09-19T07:39:05.111886+0000) 2024-09-19T07:39:24.286 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:24 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8[53845]: 2024-09-19T07:39:24.077+0000 7fddd712c640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:58.613701+0000 front 2024-09-19T07:38:58.613653+0000 (oldest deadline 2024-09-19T07:39:22.113805+0000) 2024-09-19T07:39:24.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:24 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:24.024+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:24.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:24 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:24.024+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:24.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:24 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:24.024+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:24.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:24 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:24.024+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:24.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:24 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:24.024+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:24.538 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:24 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:24.218+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:24.538 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:24 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:24.218+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:24.538 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:24 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:24.218+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:24.539 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:24 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:24.218+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:24.539 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:24 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:24.218+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.769717+0000 front 2024-09-19T07:38:51.769971+0000 (oldest deadline 2024-09-19T07:39:16.469248+0000) 2024-09-19T07:39:24.539 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:24 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:24.218+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:59.970559+0000 front 2024-09-19T07:38:59.970338+0000 (oldest deadline 2024-09-19T07:39:23.470198+0000) 2024-09-19T07:39:24.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:24 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:24.087+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:24.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:24 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:24.087+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:24.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:24 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:24.087+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:24.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:24 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:24.087+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:24.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:24 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:24.087+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:24.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:24 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:24.087+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:25.217 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:24 smithi203 podman[69490]: 2024-09-19 07:39:24.887729107 +0000 UTC m=+5.542216671 container died 2e6398352d4b2a0f83c513f0bd800da995ae7ce0b3bb75fdd7dfd12ff49057d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-19T07:39:25.218 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:24 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:24.981+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:25.218 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:24 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:24.981+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:25.218 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:24 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:24.981+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:25.218 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:24 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:24.981+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:25.218 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:24 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:24.981+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:25.526 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:25 smithi203 podman[69490]: 2024-09-19 07:39:25.316281145 +0000 UTC m=+5.970768699 container cleanup 2e6398352d4b2a0f83c513f0bd800da995ae7ce0b3bb75fdd7dfd12ff49057d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-19T07:39:25.526 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:25 smithi203 bash[69490]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8 2024-09-19T07:39:25.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:25 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:25.113+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:25.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:25 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:25.113+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:25.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:25 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:25.113+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:25.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:25 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:25.113+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:25.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:25 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:25.113+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:25.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:25 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:25.113+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:25.562 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:25 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:25.256+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:25.562 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:25 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:25.256+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:25.562 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:25 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:25.256+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:25.562 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:25 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:25.256+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:25.562 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:25 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:25.256+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.769717+0000 front 2024-09-19T07:38:51.769971+0000 (oldest deadline 2024-09-19T07:39:16.469248+0000) 2024-09-19T07:39:25.562 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:25 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:25.256+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:59.970559+0000 front 2024-09-19T07:38:59.970338+0000 (oldest deadline 2024-09-19T07:39:23.470198+0000) 2024-09-19T07:39:25.786 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:25 smithi203 podman[69507]: 2024-09-19 07:39:25.526078363 +0000 UTC m=+0.636584112 container remove 2e6398352d4b2a0f83c513f0bd800da995ae7ce0b3bb75fdd7dfd12ff49057d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:39:26.170 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:25 smithi203 podman[69593]: 2024-09-19 07:39:25.918413425 +0000 UTC m=+0.104749533 container create d83162433746fb682d6c471d810062d8aab4258377f02c81bde9bceaf56e9df9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:39:26.170 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:25 smithi203 podman[69593]: 2024-09-19 07:39:25.829802672 +0000 UTC m=+0.016138779 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:39:26.170 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:25 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:25.972+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:26.170 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:25 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:25.972+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:26.170 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:25 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:25.972+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:26.171 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:25 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:25.972+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:26.171 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:25 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:25.972+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:26.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:26 smithi203 podman[69593]: 2024-09-19 07:39:26.170656717 +0000 UTC m=+0.356992825 container init d83162433746fb682d6c471d810062d8aab4258377f02c81bde9bceaf56e9df9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T07:39:26.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:26 smithi203 podman[69593]: 2024-09-19 07:39:26.174091211 +0000 UTC m=+0.360427329 container start d83162433746fb682d6c471d810062d8aab4258377f02c81bde9bceaf56e9df9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T07:39:26.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:26 smithi203 podman[69593]: 2024-09-19 07:39:26.270704939 +0000 UTC m=+0.457041065 container attach d83162433746fb682d6c471d810062d8aab4258377f02c81bde9bceaf56e9df9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-19T07:39:26.536 INFO:journalctl@ceph.osd.8.smithi203.stdout:Sep 19 07:39:26 smithi203 podman[69593]: 2024-09-19 07:39:26.358450261 +0000 UTC m=+0.544786377 container died d83162433746fb682d6c471d810062d8aab4258377f02c81bde9bceaf56e9df9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-19T07:39:26.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:26 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:26.123+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:26.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:26 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:26.123+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:26.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:26 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:26.123+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:26.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:26 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:26.123+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:26.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:26 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:26.123+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:26.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:26 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:26.123+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:26.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:26 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:26.292+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:26.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:26 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:26.292+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:26.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:26 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:26.292+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:26.638 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:26 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:26.292+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:26.638 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:26 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:26.292+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.769717+0000 front 2024-09-19T07:38:51.769971+0000 (oldest deadline 2024-09-19T07:39:16.469248+0000) 2024-09-19T07:39:26.638 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:26 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:26.292+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:59.970559+0000 front 2024-09-19T07:38:59.970338+0000 (oldest deadline 2024-09-19T07:39:23.470198+0000) 2024-09-19T07:39:27.016 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:26 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:26.950+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:27.016 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:26 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:26.950+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:27.016 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:26 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:26.950+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:27.016 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:26 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:26.950+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:27.016 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:26 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:26.950+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:27.044 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.8.service' 2024-09-19T07:39:27.083 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:39:27.083 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-19T07:39:27.083 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-19T07:39:27.083 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.9 2024-09-19T07:39:27.250 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:27 smithi159 systemd[1]: Stopping Ceph osd.9 for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:39:27.251 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:27 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:27.249+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:27.457 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:27 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:27.092+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:27.457 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:27 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:27.092+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:27.457 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:27 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:27.092+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:27.458 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:27 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:27.092+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:27.458 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:27 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:27.092+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:27.458 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:27 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:27.092+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:27.518 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:27 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:27.249+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:27.518 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:27 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:27.249+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:27.518 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:27 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:27.249+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:27.518 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:27 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:27.249+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.769717+0000 front 2024-09-19T07:38:51.769971+0000 (oldest deadline 2024-09-19T07:39:16.469248+0000) 2024-09-19T07:39:27.518 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:27 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:27.249+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:59.970559+0000 front 2024-09-19T07:38:59.970338+0000 (oldest deadline 2024-09-19T07:39:23.470198+0000) 2024-09-19T07:39:27.886 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:27 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:27.517+0000 7f95ff800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:39:27.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:27 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:27.517+0000 7f95ff800640 -1 osd.9 104 *** Got signal Terminated *** 2024-09-19T07:39:27.887 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:27 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:27.517+0000 7f95ff800640 -1 osd.9 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:39:28.253 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:27 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:27.957+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:28.254 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:27 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:27.957+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:28.254 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:27 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:27.957+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:28.254 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:27 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:27.957+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:28.254 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:27 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:27.957+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:28.468 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:28 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:28.085+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:28.469 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:28 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:28.085+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:28.469 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:28 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:28.085+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:28.469 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:28 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:28.085+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:28.469 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:28 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:28.085+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:28.469 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:28 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:28.085+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:28.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:28 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:28.232+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:28.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:28 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:28.232+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:28.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:28 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:28.232+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:28.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:28 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:28.232+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:28.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:28 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:28.232+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.769717+0000 front 2024-09-19T07:38:51.769971+0000 (oldest deadline 2024-09-19T07:39:16.469248+0000) 2024-09-19T07:39:28.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:28 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:28.232+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:59.970559+0000 front 2024-09-19T07:38:59.970338+0000 (oldest deadline 2024-09-19T07:39:23.470198+0000) 2024-09-19T07:39:29.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:28 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:28.915+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:29.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:28 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:28.915+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:29.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:28 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:28.915+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:29.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:28 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:28.915+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:29.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:28 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:28.915+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:29.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:28 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:28.915+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:29.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:29 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:29.051+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:29.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:29 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:29.051+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:29.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:29 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:29.051+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:29.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:29 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:29.051+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:29.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:29 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:29.051+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:29.306 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:29 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:29.051+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:29.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:29 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:29.250+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:29.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:29 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:29.250+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:29.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:29 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:29.250+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:29.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:29 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:29.250+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:29.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:29 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:29.250+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.769717+0000 front 2024-09-19T07:38:51.769971+0000 (oldest deadline 2024-09-19T07:39:16.469248+0000) 2024-09-19T07:39:29.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:29 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:29.250+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:59.970559+0000 front 2024-09-19T07:38:59.970338+0000 (oldest deadline 2024-09-19T07:39:23.470198+0000) 2024-09-19T07:39:30.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:29 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:29.964+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:30.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:29 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:29.964+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:30.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:29 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:29.964+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:30.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:29 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:29.964+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:30.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:29 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:29.964+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:30.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:29 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:29.964+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:30.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:30 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:30.097+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:30.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:30 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:30.097+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:30.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:30 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:30.097+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:30.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:30 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:30.097+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:30.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:30 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:30.097+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:30.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:30 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:30.097+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:30.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:30 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:30.232+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:30.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:30 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:30.232+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:30.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:30 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:30.232+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:30.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:30 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:30.232+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:30.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:30 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:30.232+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.769717+0000 front 2024-09-19T07:38:51.769971+0000 (oldest deadline 2024-09-19T07:39:16.469248+0000) 2024-09-19T07:39:30.638 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:30 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:30.232+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:59.970559+0000 front 2024-09-19T07:38:59.970338+0000 (oldest deadline 2024-09-19T07:39:23.470198+0000) 2024-09-19T07:39:31.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:30 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:30.920+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:31.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:30 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:30.920+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:31.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:30 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:30.920+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:31.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:30 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:30.920+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:31.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:30 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:30.920+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:31.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:30 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:30.920+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:31.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:31 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:31.090+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:31.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:31 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:31.090+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:31.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:31 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:31.090+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:31.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:31 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:31.090+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:31.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:31 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:31.090+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:31.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:31 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:31.090+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:31.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:31 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:31.187+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:31.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:31 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:31.187+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:31.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:31 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:31.187+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:31.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:31 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:31.187+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:31.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:31 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:31.187+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.769717+0000 front 2024-09-19T07:38:51.769971+0000 (oldest deadline 2024-09-19T07:39:16.469248+0000) 2024-09-19T07:39:31.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:31 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:31.187+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:59.970559+0000 front 2024-09-19T07:38:59.970338+0000 (oldest deadline 2024-09-19T07:39:23.470198+0000) 2024-09-19T07:39:32.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:31 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:31.891+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:32.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:31 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:31.891+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:32.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:31 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:31.891+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:32.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:31 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:31.891+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:32.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:31 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:31.891+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:32.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:31 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:31.891+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:32.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:32 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:32.127+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:32.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:32 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:32.127+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:32.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:32 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:32.127+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:32.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:32 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:32.127+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:32.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:32 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:32.127+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:32.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:32 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:32.127+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:32.557 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:32 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:32.127+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:06.117449+0000 front 2024-09-19T07:39:06.117551+0000 (oldest deadline 2024-09-19T07:39:32.017186+0000) 2024-09-19T07:39:32.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:32.225+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:15.762029+0000 front 2024-09-19T07:38:15.762087+0000 (oldest deadline 2024-09-19T07:38:40.462265+0000) 2024-09-19T07:39:32.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:32.225+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:24.563736+0000 front 2024-09-19T07:38:24.563638+0000 (oldest deadline 2024-09-19T07:38:50.463703+0000) 2024-09-19T07:39:32.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:32.225+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.364938+0000 front 2024-09-19T07:38:33.365078+0000 (oldest deadline 2024-09-19T07:38:56.864975+0000) 2024-09-19T07:39:32.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:32.225+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.167121+0000 front 2024-09-19T07:38:43.167143+0000 (oldest deadline 2024-09-19T07:39:06.067375+0000) 2024-09-19T07:39:32.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:32.225+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.769717+0000 front 2024-09-19T07:38:51.769971+0000 (oldest deadline 2024-09-19T07:39:16.469248+0000) 2024-09-19T07:39:32.637 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:32 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9[64750]: 2024-09-19T07:39:32.225+0000 7f95fb200640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:59.970559+0000 front 2024-09-19T07:38:59.970338+0000 (oldest deadline 2024-09-19T07:39:23.470198+0000) 2024-09-19T07:39:33.182 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:32 smithi159 podman[88226]: 2024-09-19 07:39:32.889564875 +0000 UTC m=+5.518604103 container died 0f7d772f19549b88331d44faec4be260f6574faa95e4a1d448a8f56832360220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-19T07:39:33.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:32 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:32.845+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:33.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:32 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:32.845+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:33.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:32 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:32.845+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:33.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:32 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:32.845+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:33.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:32 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:32.845+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:33.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:32 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:32.845+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:33.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:33 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:33.137+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:33.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:33 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:33.137+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:33.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:33 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:33.137+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:33.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:33 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:33.137+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:33.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:33 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:33.137+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:33.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:33 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:33.137+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:33.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:33 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:33.137+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:06.117449+0000 front 2024-09-19T07:39:06.117551+0000 (oldest deadline 2024-09-19T07:39:32.017186+0000) 2024-09-19T07:39:33.599 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:33 smithi159 podman[88226]: 2024-09-19 07:39:33.339432705 +0000 UTC m=+5.968471930 container cleanup 0f7d772f19549b88331d44faec4be260f6574faa95e4a1d448a8f56832360220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T07:39:33.599 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:33 smithi159 bash[88226]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9 2024-09-19T07:39:33.886 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:33 smithi159 podman[88246]: 2024-09-19 07:39:33.598936901 +0000 UTC m=+0.695279408 container remove 0f7d772f19549b88331d44faec4be260f6574faa95e4a1d448a8f56832360220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T07:39:34.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:33 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:33.836+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:34.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:33 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:33.836+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:34.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:33 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:33.836+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:34.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:33 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:33.836+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:34.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:33 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:33.836+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:34.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:33 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:33.836+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:34.460 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:34 smithi159 podman[88332]: 2024-09-19 07:39:34.052985412 +0000 UTC m=+0.027015390 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:39:34.460 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:34 smithi159 podman[88332]: 2024-09-19 07:39:34.166488231 +0000 UTC m=+0.140518195 container create de5d29e552486e563b731e6e05a6f5e459ff021047a0b4426213173bb94d386e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9-deactivate, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:39:34.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:34 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:34.168+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:34.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:34 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:34.168+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:34.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:34 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:34.168+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:34.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:34 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:34.168+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:34.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:34 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:34.168+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:34.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:34 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:34.168+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:34.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:34 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:34.168+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:06.117449+0000 front 2024-09-19T07:39:06.117551+0000 (oldest deadline 2024-09-19T07:39:32.017186+0000) 2024-09-19T07:39:34.729 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:34 smithi159 podman[88332]: 2024-09-19 07:39:34.460859741 +0000 UTC m=+0.434889705 container init de5d29e552486e563b731e6e05a6f5e459ff021047a0b4426213173bb94d386e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T07:39:34.729 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:34 smithi159 podman[88332]: 2024-09-19 07:39:34.467690752 +0000 UTC m=+0.441720717 container start de5d29e552486e563b731e6e05a6f5e459ff021047a0b4426213173bb94d386e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T07:39:34.729 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:34 smithi159 podman[88332]: 2024-09-19 07:39:34.610816832 +0000 UTC m=+0.584846811 container attach de5d29e552486e563b731e6e05a6f5e459ff021047a0b4426213173bb94d386e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T07:39:35.096 INFO:journalctl@ceph.osd.9.smithi159.stdout:Sep 19 07:39:34 smithi159 podman[88332]: 2024-09-19 07:39:34.730097343 +0000 UTC m=+0.704127308 container died de5d29e552486e563b731e6e05a6f5e459ff021047a0b4426213173bb94d386e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-19T07:39:35.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:34 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:34.811+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:35.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:34 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:34.811+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:35.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:34 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:34.811+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:35.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:34 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:34.811+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:35.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:34 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:34.812+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:35.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:34 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:34.812+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:35.445 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.9.service' 2024-09-19T07:39:35.520 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:39:35.520 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-19T07:39:35.520 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-19T07:39:35.521 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.10 2024-09-19T07:39:35.549 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:35 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:35.172+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:35.549 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:35 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:35.172+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:35.550 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:35 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:35.172+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:35.550 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:35 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:35.172+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:35.550 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:35 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:35.172+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:35.550 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:35 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:35.172+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:35.550 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:35 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:35.172+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:06.117449+0000 front 2024-09-19T07:39:06.117551+0000 (oldest deadline 2024-09-19T07:39:32.017186+0000) 2024-09-19T07:39:35.806 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:35 smithi189 systemd[1]: Stopping Ceph osd.10 for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:39:36.168 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:35 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:35.819+0000 7fc0f4c98640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:39:36.168 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:35 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:35.819+0000 7fc0f4c98640 -1 osd.10 104 *** Got signal Terminated *** 2024-09-19T07:39:36.168 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:35 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:35.819+0000 7fc0f4c98640 -1 osd.10 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:39:36.168 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:36 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:36.167+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:36.168 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:36 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:36.167+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:36.168 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:36 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:36.167+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:36.169 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:36 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:36.167+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:36.169 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:36 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:36.167+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:36.169 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:36 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:36.167+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:36.169 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:36 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:36.167+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:06.117449+0000 front 2024-09-19T07:39:06.117551+0000 (oldest deadline 2024-09-19T07:39:32.017186+0000) 2024-09-19T07:39:36.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:35 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:35.861+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:36.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:35 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:35.861+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:36.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:35 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:35.861+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:36.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:35 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:35.861+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:36.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:35 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:35.861+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:36.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:35 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:35.861+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:37.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:36 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:36.844+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:37.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:36 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:36.844+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:37.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:36 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:36.844+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:37.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:36 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:36.844+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:37.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:36 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:36.844+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:37.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:36 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:36.844+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:37.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:37 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:37.199+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:37.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:37 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:37.199+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:37.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:37 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:37.199+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:37.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:37 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:37.199+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:37.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:37 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:37.199+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:37.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:37 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:37.199+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:37.557 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:37 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:37.199+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:06.117449+0000 front 2024-09-19T07:39:06.117551+0000 (oldest deadline 2024-09-19T07:39:32.017186+0000) 2024-09-19T07:39:38.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:37 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:37.865+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:38.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:37 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:37.865+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:38.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:37 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:37.865+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:38.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:37 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:37.865+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:38.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:37 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:37.865+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:38.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:37 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:37.865+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:38.510 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:38 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:38.156+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:38.510 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:38 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:38.156+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:38.510 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:38 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:38.156+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:38.510 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:38 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:38.156+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:38.510 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:38 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:38.156+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:38.510 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:38 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:38.156+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:38.510 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:38 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:38.156+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:06.117449+0000 front 2024-09-19T07:39:06.117551+0000 (oldest deadline 2024-09-19T07:39:32.017186+0000) 2024-09-19T07:39:39.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:38 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:38.825+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:39.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:38 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:38.825+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:39.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:38 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:38.825+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:39.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:38 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:38.825+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:39.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:38 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:38.825+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:39.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:38 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:38.825+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:39.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:39 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:39.183+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:39.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:39 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:39.183+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:39.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:39 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:39.183+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:39.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:39 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:39.183+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:39.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:39 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:39.183+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:39.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:39 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:39.183+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:39.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:39 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:39.183+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:06.117449+0000 front 2024-09-19T07:39:06.117551+0000 (oldest deadline 2024-09-19T07:39:32.017186+0000) 2024-09-19T07:39:40.247 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:39 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:39.836+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:40.247 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:39 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:39.836+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:40.247 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:39 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:39.836+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:40.247 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:39 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:39.836+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:40.247 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:39 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:39.836+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:40.247 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:39 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:39.836+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:40.248 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:39 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:39.836+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6820 osd.7 since back 2024-09-19T07:39:11.543602+0000 front 2024-09-19T07:39:11.543581+0000 (oldest deadline 2024-09-19T07:39:39.743923+0000) 2024-09-19T07:39:40.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:40 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:40.228+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:18.111319+0000 front 2024-09-19T07:38:18.111387+0000 (oldest deadline 2024-09-19T07:38:39.211150+0000) 2024-09-19T07:39:40.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:40 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:40.228+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.1 since back 2024-09-19T07:38:23.812646+0000 front 2024-09-19T07:38:23.812884+0000 (oldest deadline 2024-09-19T07:38:47.912902+0000) 2024-09-19T07:39:40.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:40 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:40.228+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:33.214019+0000 front 2024-09-19T07:38:33.213997+0000 (oldest deadline 2024-09-19T07:38:57.913908+0000) 2024-09-19T07:39:40.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:40 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:40.228+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:43.215172+0000 front 2024-09-19T07:38:43.215308+0000 (oldest deadline 2024-09-19T07:39:07.315108+0000) 2024-09-19T07:39:40.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:40 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:40.228+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.415776+0000 front 2024-09-19T07:38:51.415758+0000 (oldest deadline 2024-09-19T07:39:17.315917+0000) 2024-09-19T07:39:40.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:40 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:40.228+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.316357+0000 front 2024-09-19T07:38:57.316374+0000 (oldest deadline 2024-09-19T07:39:20.816242+0000) 2024-09-19T07:39:40.556 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:40 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10[59502]: 2024-09-19T07:39:40.228+0000 7fc0f12ae640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:06.117449+0000 front 2024-09-19T07:39:06.117551+0000 (oldest deadline 2024-09-19T07:39:32.017186+0000) 2024-09-19T07:39:41.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:40 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:40.841+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:41.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:40 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:40.841+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:41.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:40 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:40.841+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:41.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:40 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:40.841+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:41.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:40 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:40.841+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:41.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:40 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:40.841+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:41.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:40 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:40.841+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6820 osd.7 since back 2024-09-19T07:39:11.543602+0000 front 2024-09-19T07:39:11.543581+0000 (oldest deadline 2024-09-19T07:39:39.743923+0000) 2024-09-19T07:39:41.469 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:41 smithi189 podman[68506]: 2024-09-19 07:39:41.202179302 +0000 UTC m=+5.487680709 container died ee33c5f623eadbeb8c677816f39bd643de75f58591c2271540156c3730cc06a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:39:41.806 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:41 smithi189 podman[68506]: 2024-09-19 07:39:41.551939013 +0000 UTC m=+5.837440421 container cleanup ee33c5f623eadbeb8c677816f39bd643de75f58591c2271540156c3730cc06a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T07:39:41.806 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:41 smithi189 bash[68506]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10 2024-09-19T07:39:42.167 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:41 smithi189 podman[68521]: 2024-09-19 07:39:41.836235757 +0000 UTC m=+0.629633039 container remove ee33c5f623eadbeb8c677816f39bd643de75f58591c2271540156c3730cc06a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-19T07:39:42.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:41 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:41.858+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:42.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:41 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:41.858+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:42.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:41 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:41.858+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:42.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:41 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:41.858+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:42.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:41 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:41.858+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:42.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:41 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:41.858+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:42.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:41 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:41.858+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6820 osd.7 since back 2024-09-19T07:39:11.543602+0000 front 2024-09-19T07:39:11.543581+0000 (oldest deadline 2024-09-19T07:39:39.743923+0000) 2024-09-19T07:39:42.439 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:42 smithi189 podman[68608]: 2024-09-19 07:39:42.070586866 +0000 UTC m=+0.015365163 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:39:42.439 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:42 smithi189 podman[68608]: 2024-09-19 07:39:42.178733584 +0000 UTC m=+0.123511873 container create 49348b5f0e90fa466fd5141b141f15ea97e90c08219b71872b6c1f4870169033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-19T07:39:42.439 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:42 smithi189 podman[68608]: 2024-09-19 07:39:42.43954394 +0000 UTC m=+0.384322231 container init 49348b5f0e90fa466fd5141b141f15ea97e90c08219b71872b6c1f4870169033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T07:39:42.806 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:42 smithi189 podman[68608]: 2024-09-19 07:39:42.442962533 +0000 UTC m=+0.387740821 container start 49348b5f0e90fa466fd5141b141f15ea97e90c08219b71872b6c1f4870169033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T07:39:42.806 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:42 smithi189 podman[68608]: 2024-09-19 07:39:42.531338677 +0000 UTC m=+0.476116985 container attach 49348b5f0e90fa466fd5141b141f15ea97e90c08219b71872b6c1f4870169033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4) 2024-09-19T07:39:42.806 INFO:journalctl@ceph.osd.10.smithi189.stdout:Sep 19 07:39:42 smithi189 podman[68608]: 2024-09-19 07:39:42.628251285 +0000 UTC m=+0.573029573 container died 49348b5f0e90fa466fd5141b141f15ea97e90c08219b71872b6c1f4870169033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:39:43.256 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.10.service' 2024-09-19T07:39:43.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:42 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:42.851+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:43.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:42 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:42.851+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:43.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:42 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:42.851+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:43.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:42 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:42.851+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:43.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:42 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:42.851+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:43.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:42 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:42.851+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:43.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:42 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:42.851+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6820 osd.7 since back 2024-09-19T07:39:11.543602+0000 front 2024-09-19T07:39:11.543581+0000 (oldest deadline 2024-09-19T07:39:39.743923+0000) 2024-09-19T07:39:43.335 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:39:43.335 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-19T07:39:43.335 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-19T07:39:43.336 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.11 2024-09-19T07:39:43.726 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:43 smithi203 systemd[1]: Stopping Ceph osd.11 for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:39:44.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:43 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:43.726+0000 7f88776ed640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:39:44.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:43 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:43.726+0000 7f88776ed640 -1 osd.11 104 *** Got signal Terminated *** 2024-09-19T07:39:44.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:43 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:43.726+0000 7f88776ed640 -1 osd.11 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:39:44.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:43 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:43.823+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:44.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:43 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:43.823+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:44.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:43 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:43.823+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:44.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:43 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:43.823+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:44.037 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:43 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:43.823+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:44.037 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:43 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:43.823+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:44.037 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:43 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:43.823+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6820 osd.7 since back 2024-09-19T07:39:11.543602+0000 front 2024-09-19T07:39:11.543581+0000 (oldest deadline 2024-09-19T07:39:39.743923+0000) 2024-09-19T07:39:45.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:44 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:44.819+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:45.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:44 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:44.819+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:45.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:44 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:44.819+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:45.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:44 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:44.819+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:45.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:44 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:44.819+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:45.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:44 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:44.819+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:45.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:44 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:44.819+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6820 osd.7 since back 2024-09-19T07:39:11.543602+0000 front 2024-09-19T07:39:11.543581+0000 (oldest deadline 2024-09-19T07:39:39.743923+0000) 2024-09-19T07:39:46.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:45 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:45.814+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:46.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:45 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:45.814+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:46.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:45 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:45.814+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:46.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:45 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:45.814+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:46.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:45 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:45.814+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:46.286 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:45 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:45.814+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:46.287 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:45 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:45.814+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6820 osd.7 since back 2024-09-19T07:39:11.543602+0000 front 2024-09-19T07:39:11.543581+0000 (oldest deadline 2024-09-19T07:39:39.743923+0000) 2024-09-19T07:39:47.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:46 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:46.765+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:47.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:46 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:46.765+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:47.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:46 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:46.765+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:47.037 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:46 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:46.765+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:47.037 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:46 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:46.765+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:47.037 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:46 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:46.765+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:47.037 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:46 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:46.765+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6820 osd.7 since back 2024-09-19T07:39:11.543602+0000 front 2024-09-19T07:39:11.543581+0000 (oldest deadline 2024-09-19T07:39:39.743923+0000) 2024-09-19T07:39:48.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:47 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:47.738+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6806 osd.0 since back 2024-09-19T07:38:14.134062+0000 front 2024-09-19T07:38:14.134033+0000 (oldest deadline 2024-09-19T07:38:40.033871+0000) 2024-09-19T07:39:48.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:47 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:47.738+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-09-19T07:38:34.536815+0000 front 2024-09-19T07:38:34.537022+0000 (oldest deadline 2024-09-19T07:38:58.037065+0000) 2024-09-19T07:39:48.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:47 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:47.738+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6814 osd.3 since back 2024-09-19T07:38:38.037773+0000 front 2024-09-19T07:38:38.037704+0000 (oldest deadline 2024-09-19T07:39:05.638167+0000) 2024-09-19T07:39:48.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:47 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:47.738+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.4 since back 2024-09-19T07:38:51.340213+0000 front 2024-09-19T07:38:51.340200+0000 (oldest deadline 2024-09-19T07:39:14.840171+0000) 2024-09-19T07:39:48.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:47 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:47.738+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2024-09-19T07:38:57.041198+0000 front 2024-09-19T07:38:57.041263+0000 (oldest deadline 2024-09-19T07:39:21.741287+0000) 2024-09-19T07:39:48.037 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:47 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:47.738+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.159:6822 osd.6 since back 2024-09-19T07:39:02.842289+0000 front 2024-09-19T07:39:02.842250+0000 (oldest deadline 2024-09-19T07:39:28.742088+0000) 2024-09-19T07:39:48.037 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:47 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11[60419]: 2024-09-19T07:39:47.738+0000 7f886fcfb640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6820 osd.7 since back 2024-09-19T07:39:11.543602+0000 front 2024-09-19T07:39:11.543581+0000 (oldest deadline 2024-09-19T07:39:39.743923+0000) 2024-09-19T07:39:49.410 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:49 smithi203 podman[69709]: 2024-09-19 07:39:49.132394781 +0000 UTC m=+5.594599297 container died 9d16939d78d34377b7ab2d4b6ec1bde286fd87dcf0fd12c3882dbbdf2fd2240c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-19T07:39:49.760 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:49 smithi203 podman[69709]: 2024-09-19 07:39:49.492633141 +0000 UTC m=+5.954837646 container cleanup 9d16939d78d34377b7ab2d4b6ec1bde286fd87dcf0fd12c3882dbbdf2fd2240c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-19T07:39:49.760 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:49 smithi203 bash[69709]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11 2024-09-19T07:39:50.036 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:49 smithi203 podman[69725]: 2024-09-19 07:39:49.760336946 +0000 UTC m=+0.625958909 container remove 9d16939d78d34377b7ab2d4b6ec1bde286fd87dcf0fd12c3882dbbdf2fd2240c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2) 2024-09-19T07:39:50.404 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:50 smithi203 podman[69817]: 2024-09-19 07:39:49.996672643 +0000 UTC m=+0.015653409 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:39:50.404 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:50 smithi203 podman[69817]: 2024-09-19 07:39:50.102748842 +0000 UTC m=+0.121729596 container create bfc5fb5d933836e3601b4c6216349a6c38dc272788519c6d3259b8213b7116bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-19T07:39:50.405 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:50 smithi203 podman[69817]: 2024-09-19 07:39:50.321928012 +0000 UTC m=+0.340908766 container init bfc5fb5d933836e3601b4c6216349a6c38dc272788519c6d3259b8213b7116bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11-deactivate, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T07:39:50.405 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:50 smithi203 podman[69817]: 2024-09-19 07:39:50.325117355 +0000 UTC m=+0.344098119 container start bfc5fb5d933836e3601b4c6216349a6c38dc272788519c6d3259b8213b7116bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T07:39:50.405 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:50 smithi203 podman[69817]: 2024-09-19 07:39:50.405037252 +0000 UTC m=+0.424018023 container attach bfc5fb5d933836e3601b4c6216349a6c38dc272788519c6d3259b8213b7116bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T07:39:50.786 INFO:journalctl@ceph.osd.11.smithi203.stdout:Sep 19 07:39:50 smithi203 podman[69817]: 2024-09-19 07:39:50.513943541 +0000 UTC m=+0.532924316 container died bfc5fb5d933836e3601b4c6216349a6c38dc272788519c6d3259b8213b7116bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-19T07:39:51.083 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@osd.11.service' 2024-09-19T07:39:51.121 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:39:51.121 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-19T07:39:51.121 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-19T07:39:51.121 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.a 2024-09-19T07:39:51.552 INFO:journalctl@ceph.mds.a.smithi159.stdout:Sep 19 07:39:51 smithi159 systemd[1]: Stopping Ceph mds.a for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:39:51.886 INFO:journalctl@ceph.mds.a.smithi159.stdout:Sep 19 07:39:51 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-a[71700]: 2024-09-19T07:39:51.551+0000 7fe092e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:39:51.887 INFO:journalctl@ceph.mds.a.smithi159.stdout:Sep 19 07:39:51 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-a[71700]: 2024-09-19T07:39:51.551+0000 7fe092e00640 -1 mds.a *** got signal Terminated *** 2024-09-19T07:39:55.317 INFO:journalctl@ceph.mds.a.smithi159.stdout:Sep 19 07:39:55 smithi159 podman[88433]: 2024-09-19 07:39:55.050490721 +0000 UTC m=+3.642191124 container died 202929e4eb2d165a1c5a64c0255cef401a47932b8794c791b09dde1e0370f4ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T07:39:55.637 INFO:journalctl@ceph.mds.a.smithi159.stdout:Sep 19 07:39:55 smithi159 podman[88433]: 2024-09-19 07:39:55.447974325 +0000 UTC m=+4.039674721 container cleanup 202929e4eb2d165a1c5a64c0255cef401a47932b8794c791b09dde1e0370f4ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:39:55.637 INFO:journalctl@ceph.mds.a.smithi159.stdout:Sep 19 07:39:55 smithi159 bash[88433]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-a 2024-09-19T07:39:56.084 INFO:journalctl@ceph.mds.a.smithi159.stdout:Sep 19 07:39:55 smithi159 podman[88444]: 2024-09-19 07:39:55.766495342 +0000 UTC m=+0.708913385 container remove 202929e4eb2d165a1c5a64c0255cef401a47932b8794c791b09dde1e0370f4ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-19T07:39:56.104 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.a.service' 2024-09-19T07:39:56.175 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:39:56.175 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-19T07:39:56.176 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-19T07:39:56.176 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.d 2024-09-19T07:39:56.387 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:39:56 smithi159 systemd[1]: Stopping Ceph mds.d for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:39:56.887 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:39:56 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-d[70540]: 2024-09-19T07:39:56.571+0000 7fea28c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:39:56.887 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:39:56 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-d[70540]: 2024-09-19T07:39:56.571+0000 7fea28c00640 -1 mds.d *** got signal Terminated *** 2024-09-19T07:40:00.887 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:40:00 smithi159 podman[88553]: 2024-09-19 07:40:00.586015983 +0000 UTC m=+4.126385163 container died 637c6bb5df7e439b32a6d5072ac29026a1e734af8ae270f98c92c80947549e1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-19T07:40:01.270 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:40:01 smithi159 podman[88553]: 2024-09-19 07:40:01.085542444 +0000 UTC m=+4.625911624 container cleanup 637c6bb5df7e439b32a6d5072ac29026a1e734af8ae270f98c92c80947549e1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T07:40:01.270 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:40:01 smithi159 bash[88553]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-d 2024-09-19T07:40:01.523 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.d.service' 2024-09-19T07:40:01.572 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:40:01 smithi159 podman[88578]: 2024-09-19 07:40:01.2701442 +0000 UTC m=+0.668355153 container remove 637c6bb5df7e439b32a6d5072ac29026a1e734af8ae270f98c92c80947549e1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-19T07:40:01.572 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:40:01 smithi159 systemd[1]: ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.d.service: Deactivated successfully. 2024-09-19T07:40:01.572 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:40:01 smithi159 systemd[1]: Stopped Ceph mds.d for c9d18048-7657-11ef-bceb-c7b262605968. 2024-09-19T07:40:01.572 INFO:journalctl@ceph.mds.d.smithi159.stdout:Sep 19 07:40:01 smithi159 systemd[1]: ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.d.service: Consumed 1.451s CPU time. 2024-09-19T07:40:01.597 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:40:01.598 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-19T07:40:01.598 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-19T07:40:01.598 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.g 2024-09-19T07:40:01.887 INFO:journalctl@ceph.mds.g.smithi159.stdout:Sep 19 07:40:01 smithi159 systemd[1]: Stopping Ceph mds.g for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:40:02.386 INFO:journalctl@ceph.mds.g.smithi159.stdout:Sep 19 07:40:02 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-g[71294]: 2024-09-19T07:40:02.075+0000 7feb10600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:40:02.387 INFO:journalctl@ceph.mds.g.smithi159.stdout:Sep 19 07:40:02 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-g[71294]: 2024-09-19T07:40:02.075+0000 7feb10600640 -1 mds.g *** got signal Terminated *** 2024-09-19T07:40:06.387 INFO:journalctl@ceph.mds.g.smithi159.stdout:Sep 19 07:40:06 smithi159 podman[88683]: 2024-09-19 07:40:06.091049119 +0000 UTC m=+4.173258275 container died 9ef3eb9dc55758e9793aba8d7e4772d0f06117724a636dd65430fd75bfd07441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-g, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T07:40:06.790 INFO:journalctl@ceph.mds.g.smithi159.stdout:Sep 19 07:40:06 smithi159 podman[88683]: 2024-09-19 07:40:06.50588539 +0000 UTC m=+4.588094545 container cleanup 9ef3eb9dc55758e9793aba8d7e4772d0f06117724a636dd65430fd75bfd07441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T07:40:06.790 INFO:journalctl@ceph.mds.g.smithi159.stdout:Sep 19 07:40:06 smithi159 bash[88683]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-g 2024-09-19T07:40:07.042 INFO:journalctl@ceph.mds.g.smithi159.stdout:Sep 19 07:40:06 smithi159 podman[88696]: 2024-09-19 07:40:06.790463837 +0000 UTC m=+0.691357764 container remove 9ef3eb9dc55758e9793aba8d7e4772d0f06117724a636dd65430fd75bfd07441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-g, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-19T07:40:07.064 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.g.service' 2024-09-19T07:40:07.138 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:40:07.138 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-19T07:40:07.138 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-19T07:40:07.138 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.j 2024-09-19T07:40:07.387 INFO:journalctl@ceph.mds.j.smithi159.stdout:Sep 19 07:40:07 smithi159 systemd[1]: Stopping Ceph mds.j for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:40:07.886 INFO:journalctl@ceph.mds.j.smithi159.stdout:Sep 19 07:40:07 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-j[70156]: 2024-09-19T07:40:07.495+0000 7fcdabc00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:40:07.887 INFO:journalctl@ceph.mds.j.smithi159.stdout:Sep 19 07:40:07 smithi159 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-j[70156]: 2024-09-19T07:40:07.495+0000 7fcdabc00640 -1 mds.j *** got signal Terminated *** 2024-09-19T07:40:09.597 INFO:journalctl@ceph.mds.j.smithi159.stdout:Sep 19 07:40:09 smithi159 podman[88804]: 2024-09-19 07:40:09.258874086 +0000 UTC m=+1.853928460 container died 235cea9acf8e8d74acab68424ea21a44529506a364ccf54f6bdc98d896d9b51f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T07:40:09.887 INFO:journalctl@ceph.mds.j.smithi159.stdout:Sep 19 07:40:09 smithi159 podman[88804]: 2024-09-19 07:40:09.717812204 +0000 UTC m=+2.312866583 container cleanup 235cea9acf8e8d74acab68424ea21a44529506a364ccf54f6bdc98d896d9b51f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-j, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T07:40:09.887 INFO:journalctl@ceph.mds.j.smithi159.stdout:Sep 19 07:40:09 smithi159 bash[88804]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-j 2024-09-19T07:40:10.224 INFO:journalctl@ceph.mds.j.smithi159.stdout:Sep 19 07:40:09 smithi159 podman[88815]: 2024-09-19 07:40:09.958658768 +0000 UTC m=+0.680413993 container remove 235cea9acf8e8d74acab68424ea21a44529506a364ccf54f6bdc98d896d9b51f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-j, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-19T07:40:10.244 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.j.service' 2024-09-19T07:40:10.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:40:10.313 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-19T07:40:10.314 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-19T07:40:10.314 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.b 2024-09-19T07:40:10.596 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:40:10 smithi189 systemd[1]: Stopping Ceph mds.b for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:40:11.052 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:40:10 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-b[64691]: 2024-09-19T07:40:10.693+0000 7f39ff6cb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:40:11.052 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:40:10 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-b[64691]: 2024-09-19T07:40:10.693+0000 7f39ff6cb640 -1 mds.b *** got signal Terminated *** 2024-09-19T07:40:13.410 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:40:13 smithi189 podman[68710]: 2024-09-19 07:40:13.138733627 +0000 UTC m=+2.628027443 container died 26d2b7f256a54bc9627232e16eb9f0bb643171be123cd282bd0b00b980cbd823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T07:40:13.760 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:40:13 smithi189 podman[68710]: 2024-09-19 07:40:13.518371351 +0000 UTC m=+3.007665165 container cleanup 26d2b7f256a54bc9627232e16eb9f0bb643171be123cd282bd0b00b980cbd823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-b, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T07:40:13.761 INFO:journalctl@ceph.mds.b.smithi189.stdout:Sep 19 07:40:13 smithi189 bash[68710]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-b 2024-09-19T07:40:13.908 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.b.service' 2024-09-19T07:40:13.944 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:40:13.945 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-19T07:40:13.945 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-19T07:40:13.945 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.e 2024-09-19T07:40:14.281 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:40:14 smithi189 systemd[1]: Stopping Ceph mds.e for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:40:14.555 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:40:14 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-e[64090]: 2024-09-19T07:40:14.280+0000 7fada4de5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:40:14.556 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:40:14 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-e[64090]: 2024-09-19T07:40:14.280+0000 7fada4de5640 -1 mds.e *** got signal Terminated *** 2024-09-19T07:40:17.398 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:40:17 smithi189 podman[68843]: 2024-09-19 07:40:17.131353958 +0000 UTC m=+2.965615359 container died aa4b2d77bfa0e76f040f12c4e626b60532b4b6d8ff41d072311ff571f1896856 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-e, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:40:17.781 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:40:17 smithi189 podman[68843]: 2024-09-19 07:40:17.48033416 +0000 UTC m=+3.314595562 container cleanup aa4b2d77bfa0e76f040f12c4e626b60532b4b6d8ff41d072311ff571f1896856 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T07:40:17.781 INFO:journalctl@ceph.mds.e.smithi189.stdout:Sep 19 07:40:17 smithi189 bash[68843]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-e 2024-09-19T07:40:17.966 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.e.service' 2024-09-19T07:40:17.998 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:40:17.999 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-19T07:40:17.999 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-19T07:40:17.999 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.h 2024-09-19T07:40:18.334 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:40:18 smithi189 systemd[1]: Stopping Ceph mds.h for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:40:18.806 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:40:18 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-h[64391]: 2024-09-19T07:40:18.333+0000 7fa3ad8eb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:40:18.806 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:40:18 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-h[64391]: 2024-09-19T07:40:18.333+0000 7fa3ad8eb640 -1 mds.h *** got signal Terminated *** 2024-09-19T07:40:21.393 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:40:21 smithi189 podman[68963]: 2024-09-19 07:40:21.130995689 +0000 UTC m=+2.909513014 container died 806c9ecaa4a36bceb933560556c8690a77e4fd014a7e4f50fd4e68f282937022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-h, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T07:40:21.777 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:40:21 smithi189 podman[68963]: 2024-09-19 07:40:21.492826909 +0000 UTC m=+3.271344242 container cleanup 806c9ecaa4a36bceb933560556c8690a77e4fd014a7e4f50fd4e68f282937022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-h, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-19T07:40:21.777 INFO:journalctl@ceph.mds.h.smithi189.stdout:Sep 19 07:40:21 smithi189 bash[68963]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-h 2024-09-19T07:40:21.909 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.h.service' 2024-09-19T07:40:21.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:40:21.942 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-19T07:40:21.942 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-19T07:40:21.942 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.k 2024-09-19T07:40:22.055 INFO:journalctl@ceph.mds.k.smithi189.stdout:Sep 19 07:40:22 smithi189 systemd[1]: Stopping Ceph mds.k for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:40:22.556 INFO:journalctl@ceph.mds.k.smithi189.stdout:Sep 19 07:40:22 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-k[63775]: 2024-09-19T07:40:22.254+0000 7fcc3dbab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:40:22.556 INFO:journalctl@ceph.mds.k.smithi189.stdout:Sep 19 07:40:22 smithi189 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-k[63775]: 2024-09-19T07:40:22.254+0000 7fcc3dbab640 -1 mds.k *** got signal Terminated *** 2024-09-19T07:40:25.672 INFO:journalctl@ceph.mds.k.smithi189.stdout:Sep 19 07:40:25 smithi189 podman[69085]: 2024-09-19 07:40:25.317589923 +0000 UTC m=+3.165058393 container died d03627f586fa3ee2989b402a8a8d4d7e2fa72966577f92ba6eef63d88d71d427 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T07:40:26.055 INFO:journalctl@ceph.mds.k.smithi189.stdout:Sep 19 07:40:25 smithi189 podman[69085]: 2024-09-19 07:40:25.771362463 +0000 UTC m=+3.618830933 container cleanup d03627f586fa3ee2989b402a8a8d4d7e2fa72966577f92ba6eef63d88d71d427 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-k, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:40:26.055 INFO:journalctl@ceph.mds.k.smithi189.stdout:Sep 19 07:40:25 smithi189 bash[69085]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-k 2024-09-19T07:40:26.189 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.k.service' 2024-09-19T07:40:26.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:40:26.221 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-19T07:40:26.221 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-19T07:40:26.222 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.c 2024-09-19T07:40:26.524 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:40:26 smithi203 systemd[1]: Stopping Ceph mds.c for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:40:26.788 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:40:26 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-c[65479]: 2024-09-19T07:40:26.524+0000 7f9ebb362640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:40:26.789 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:40:26 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-c[65479]: 2024-09-19T07:40:26.524+0000 7f9ebb362640 -1 mds.c *** got signal Terminated *** 2024-09-19T07:40:29.394 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:40:29 smithi203 podman[69932]: 2024-09-19 07:40:29.139450064 +0000 UTC m=+2.723567546 container died 1e80ff8e50531c7aa0101a6a4f85cdd749497210d6c14d93b3ab1483736aa293 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T07:40:29.768 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:40:29 smithi203 podman[69932]: 2024-09-19 07:40:29.49274453 +0000 UTC m=+3.076862053 container cleanup 1e80ff8e50531c7aa0101a6a4f85cdd749497210d6c14d93b3ab1483736aa293 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-c, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T07:40:29.768 INFO:journalctl@ceph.mds.c.smithi203.stdout:Sep 19 07:40:29 smithi203 bash[69932]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-c 2024-09-19T07:40:29.921 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.c.service' 2024-09-19T07:40:29.956 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:40:29.957 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-19T07:40:29.957 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-19T07:40:29.957 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.f 2024-09-19T07:40:30.280 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:40:30 smithi203 systemd[1]: Stopping Ceph mds.f for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:40:30.536 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:40:30 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-f[65177]: 2024-09-19T07:40:30.279+0000 7fdec6b10640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:40:30.536 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:40:30 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-f[65177]: 2024-09-19T07:40:30.279+0000 7fdec6b10640 -1 mds.f *** got signal Terminated *** 2024-09-19T07:40:33.430 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:40:33 smithi203 podman[70054]: 2024-09-19 07:40:33.136351784 +0000 UTC m=+2.961998022 container died fc71e70c06dd2955171cb692fc7b07f8d74d422f69e071c9873f376acb49a22b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-f, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T07:40:33.781 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:40:33 smithi203 podman[70054]: 2024-09-19 07:40:33.546631267 +0000 UTC m=+3.372277509 container cleanup fc71e70c06dd2955171cb692fc7b07f8d74d422f69e071c9873f376acb49a22b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:40:33.781 INFO:journalctl@ceph.mds.f.smithi203.stdout:Sep 19 07:40:33 smithi203 bash[70054]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-f 2024-09-19T07:40:33.915 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.f.service' 2024-09-19T07:40:33.947 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:40:33.947 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-19T07:40:33.948 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-19T07:40:33.948 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.i 2024-09-19T07:40:34.036 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:40:34 smithi203 systemd[1]: Stopping Ceph mds.i for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:40:34.536 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:40:34 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-i[65781]: 2024-09-19T07:40:34.266+0000 7fe433464640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:40:34.536 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:40:34 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-i[65781]: 2024-09-19T07:40:34.266+0000 7fe433464640 -1 mds.i *** got signal Terminated *** 2024-09-19T07:40:37.434 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:40:37 smithi203 podman[70174]: 2024-09-19 07:40:37.1265564 +0000 UTC m=+2.962310598 container died 826b5b583f2d7345b1b6779772f18f337edf23344c7056e71d0d679143d2cd36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T07:40:37.751 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:40:37 smithi203 podman[70174]: 2024-09-19 07:40:37.517584791 +0000 UTC m=+3.353338989 container cleanup 826b5b583f2d7345b1b6779772f18f337edf23344c7056e71d0d679143d2cd36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-i, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:40:37.752 INFO:journalctl@ceph.mds.i.smithi203.stdout:Sep 19 07:40:37 smithi203 bash[70174]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-i 2024-09-19T07:40:37.904 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.i.service' 2024-09-19T07:40:37.936 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:40:37.936 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-19T07:40:37.936 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-19T07:40:37.936 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.l 2024-09-19T07:40:38.286 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 19 07:40:38 smithi203 systemd[1]: Stopping Ceph mds.l for c9d18048-7657-11ef-bceb-c7b262605968... 2024-09-19T07:40:38.286 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 19 07:40:38 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-l[64876]: 2024-09-19T07:40:38.253+0000 7fa55b28a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:40:38.286 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 19 07:40:38 smithi203 ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-l[64876]: 2024-09-19T07:40:38.253+0000 7fa55b28a640 -1 mds.l *** got signal Terminated *** 2024-09-19T07:40:42.536 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 19 07:40:42 smithi203 podman[70292]: 2024-09-19 07:40:42.260670192 +0000 UTC m=+4.112580571 container died 8c453e9aff1e540afd209aea06960c818c3a7c37068f95b816e6a55fe551c735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-19T07:40:42.789 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 19 07:40:42 smithi203 podman[70292]: 2024-09-19 07:40:42.546259388 +0000 UTC m=+4.398169757 container cleanup 8c453e9aff1e540afd209aea06960c818c3a7c37068f95b816e6a55fe551c735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-l, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T07:40:42.789 INFO:journalctl@ceph.mds.l.smithi203.stdout:Sep 19 07:40:42 smithi203 bash[70292]: ceph-c9d18048-7657-11ef-bceb-c7b262605968-mds-l 2024-09-19T07:40:42.914 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c9d18048-7657-11ef-bceb-c7b262605968@mds.l.service' 2024-09-19T07:40:42.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:40:42.946 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-19T07:40:42.946 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9d18048-7657-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T07:40:43.229 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:40:47.203 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9d18048-7657-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T07:40:47.407 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:40:49.984 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9d18048-7657-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T07:40:50.187 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:40:52.593 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:40:52.658 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:40:52.691 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:40:52.718 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T07:40:52.720 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911794/remote/smithi159/crash 2024-09-19T07:40:52.721 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/crash -- . 2024-09-19T07:40:52.787 INFO:teuthology.orchestra.run.smithi159.stderr:tar: /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T07:40:52.787 INFO:teuthology.orchestra.run.smithi159.stderr:tar: Error is not recoverable: exiting now 2024-09-19T07:40:52.790 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911794/remote/smithi189/crash 2024-09-19T07:40:52.791 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/crash -- . 2024-09-19T07:40:52.827 INFO:teuthology.orchestra.run.smithi189.stderr:tar: /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T07:40:52.827 INFO:teuthology.orchestra.run.smithi189.stderr:tar: Error is not recoverable: exiting now 2024-09-19T07:40:52.829 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911794/remote/smithi203/crash 2024-09-19T07:40:52.830 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/crash -- . 2024-09-19T07:40:52.869 INFO:teuthology.orchestra.run.smithi203.stderr:tar: /var/lib/ceph/c9d18048-7657-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T07:40:52.869 INFO:teuthology.orchestra.run.smithi203.stderr:tar: Error is not recoverable: exiting now 2024-09-19T07:40:52.870 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T07:40:52.870 DEBUG:teuthology.orchestra.run.smithi159:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-19T07:40:52.949 INFO:tasks.cephadm:Compressing logs... 2024-09-19T07:40:52.949 DEBUG:teuthology.orchestra.run.smithi159:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T07:40:52.954 DEBUG:teuthology.orchestra.run.smithi189:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T07:40:52.956 DEBUG:teuthology.orchestra.run.smithi203:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T07:40:52.981 INFO:teuthology.orchestra.run.smithi203.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T07:40:52.981 INFO:teuthology.orchestra.run.smithi203.stderr:: No such file or directory 2024-09-19T07:40:52.981 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T07:40:52.981 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-19T07:40:52.981 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-19T07:40:52.983 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T07:40:52.983 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-19T07:40:52.984 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mon.c.log: /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T07:40:52.984 INFO:teuthology.orchestra.run.smithi203.stderr: 82.7% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T07:40:52.984 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T07:40:52.984 INFO:teuthology.orchestra.run.smithi203.stderr:gzip/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.2.log: -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.log 2024-09-19T07:40:52.984 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.5.log: --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-19T07:40:52.984 INFO:teuthology.orchestra.run.smithi203.stderr: 90.4% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T07:40:52.991 INFO:teuthology.orchestra.run.smithi189.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-19T07:40:52.992 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-19T07:40:52.992 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T07:40:52.992 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T07:40:52.992 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T07:40:52.992 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-19T07:40:52.992 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-19T07:40:52.992 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T07:40:52.992 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.b.log: /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T07:40:52.992 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T07:40:52.992 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.audit.log: 94.7% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-19T07:40:52.992 INFO:teuthology.orchestra.run.smithi189.stderr: 90.4% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T07:40:52.993 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-19T07:40:52.993 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- 82.8% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T07:40:52.993 INFO:teuthology.orchestra.run.smithi189.stderr: /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.log 2024-09-19T07:40:52.993 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-19T07:40:52.994 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.log: 86.5% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T07:40:52.994 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-19T07:40:52.995 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-19T07:40:52.995 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.8.log: /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.l.log: 94.8% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-19T07:40:52.995 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-19T07:40:52.995 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mgr.z.log: 90.8% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T07:40:52.995 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.log: /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mgr.y.log: 86.5% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T07:40:52.996 INFO:teuthology.orchestra.run.smithi203.stderr: 92.5% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-19T07:40:52.996 INFO:teuthology.orchestra.run.smithi189.stderr: 92.4% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-19T07:40:52.997 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-19T07:40:52.997 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T07:40:52.999 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-19T07:40:52.999 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-19T07:40:52.999 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mon.b.log: 90.8% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T07:40:53.004 INFO:teuthology.orchestra.run.smithi159.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log: No such file or directory 2024-09-19T07:40:53.004 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-09-19T07:40:53.004 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-19T07:40:53.005 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-19T07:40:53.006 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T07:40:53.007 INFO:teuthology.orchestra.run.smithi159.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-19T07:40:53.007 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T07:40:53.007 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-19T07:40:53.009 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T07:40:53.010 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-19T07:40:53.011 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T07:40:53.012 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-19T07:40:53.012 INFO:teuthology.orchestra.run.smithi159.stderr: 82.7% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T07:40:53.012 INFO:teuthology.orchestra.run.smithi159.stderr: 90.2% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T07:40:53.013 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T07:40:53.013 INFO:teuthology.orchestra.run.smithi159.stderr: 94.7% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-19T07:40:53.014 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.log 2024-09-19T07:40:53.015 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-19T07:40:53.018 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.log: /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.d.log: 94.7% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-19T07:40:53.019 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-19T07:40:53.019 INFO:teuthology.orchestra.run.smithi159.stderr: 86.5% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T07:40:53.021 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-19T07:40:53.021 INFO:teuthology.orchestra.run.smithi159.stderr: 94.6% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-19T07:40:53.023 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T07:40:53.023 INFO:teuthology.orchestra.run.smithi203.stderr: 92.6% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-19T07:40:53.025 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose 90.9% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T07:40:53.025 INFO:teuthology.orchestra.run.smithi159.stderr: -- /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T07:40:53.025 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T07:40:53.025 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T07:40:53.027 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.8% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-19T07:40:53.035 INFO:teuthology.orchestra.run.smithi159.stderr: 93.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T07:40:53.039 INFO:teuthology.orchestra.run.smithi189.stderr: 92.8% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-19T07:40:53.039 INFO:teuthology.orchestra.run.smithi203.stderr: 94.1% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-19T07:40:53.053 INFO:teuthology.orchestra.run.smithi189.stderr: 94.0% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-19T07:40:53.066 INFO:teuthology.orchestra.run.smithi203.stderr: 93.3% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-19T07:40:53.185 INFO:teuthology.orchestra.run.smithi203.stderr: 91.7% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-19T07:40:53.225 INFO:teuthology.orchestra.run.smithi189.stderr: 92.7% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-19T07:40:53.251 INFO:teuthology.orchestra.run.smithi159.stderr: 90.2% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-19T07:40:53.291 INFO:teuthology.orchestra.run.smithi189.stderr: 91.8% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-19T07:40:53.922 INFO:teuthology.orchestra.run.smithi159.stderr: 90.5% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-19T07:40:56.437 INFO:teuthology.orchestra.run.smithi203.stderr: 93.0% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-19T07:40:56.713 INFO:teuthology.orchestra.run.smithi189.stderr: 92.7% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-19T07:40:56.800 INFO:teuthology.orchestra.run.smithi203.stderr: 92.9% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-19T07:40:57.177 INFO:teuthology.orchestra.run.smithi189.stderr: 92.8% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T07:40:57.425 INFO:teuthology.orchestra.run.smithi203.stderr: 92.7% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T07:40:57.811 INFO:teuthology.orchestra.run.smithi189.stderr: 92.8% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T07:40:58.417 INFO:teuthology.orchestra.run.smithi159.stderr: 92.9% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T07:40:58.451 INFO:teuthology.orchestra.run.smithi189.stderr: 92.7% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T07:40:58.453 INFO:teuthology.orchestra.run.smithi189.stderr: 2024-09-19T07:40:58.453 INFO:teuthology.orchestra.run.smithi189.stderr:real 0m5.482s 2024-09-19T07:40:58.453 INFO:teuthology.orchestra.run.smithi189.stderr:user 0m18.185s 2024-09-19T07:40:58.453 INFO:teuthology.orchestra.run.smithi189.stderr:sys 0m0.675s 2024-09-19T07:40:58.557 INFO:teuthology.orchestra.run.smithi159.stderr: 92.8% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T07:40:58.649 INFO:teuthology.orchestra.run.smithi159.stderr: 92.9% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T07:40:59.068 INFO:teuthology.orchestra.run.smithi203.stderr: 92.8% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T07:40:59.070 INFO:teuthology.orchestra.run.smithi203.stderr: 2024-09-19T07:40:59.070 INFO:teuthology.orchestra.run.smithi203.stderr:real 0m6.099s 2024-09-19T07:40:59.071 INFO:teuthology.orchestra.run.smithi203.stderr:user 0m17.532s 2024-09-19T07:40:59.071 INFO:teuthology.orchestra.run.smithi203.stderr:sys 0m0.641s 2024-09-19T07:40:59.547 INFO:teuthology.orchestra.run.smithi159.stderr: 92.8% -- replaced with /var/log/ceph/c9d18048-7657-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-19T07:40:59.550 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-09-19T07:40:59.550 INFO:teuthology.orchestra.run.smithi159.stderr:real 0m6.568s 2024-09-19T07:40:59.550 INFO:teuthology.orchestra.run.smithi159.stderr:user 0m22.200s 2024-09-19T07:40:59.550 INFO:teuthology.orchestra.run.smithi159.stderr:sys 0m2.022s 2024-09-19T07:40:59.551 INFO:tasks.cephadm:Archiving logs... 2024-09-19T07:40:59.552 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911794/remote/smithi159/log 2024-09-19T07:40:59.553 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T07:41:03.092 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911794/remote/smithi189/log 2024-09-19T07:41:03.093 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T07:41:06.088 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911794/remote/smithi203/log 2024-09-19T07:41:06.089 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T07:41:09.050 INFO:tasks.cephadm:Removing cluster... 2024-09-19T07:41:09.050 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9d18048-7657-11ef-bceb-c7b262605968 --force 2024-09-19T07:41:09.322 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:41:09.832 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9d18048-7657-11ef-bceb-c7b262605968 --force 2024-09-19T07:41:10.036 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:41:10.394 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c9d18048-7657-11ef-bceb-c7b262605968 --force 2024-09-19T07:41:10.595 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: c9d18048-7657-11ef-bceb-c7b262605968 2024-09-19T07:41:10.881 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T07:41:10.881 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T07:41:10.922 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T07:41:10.939 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T07:41:10.954 INFO:tasks.cephadm:Teardown complete 2024-09-19T07:41:10.955 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T07:41:10.966 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T07:41:10.967 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T07:41:10.971 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T07:41:10.981 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T07:41:11.092 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T07:41:11.102 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T07:41:11.102 DEBUG:teuthology.orchestra.run.smithi159:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:41:11.107 DEBUG:teuthology.orchestra.run.smithi189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:41:11.109 DEBUG:teuthology.orchestra.run.smithi203:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:41:11.122 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:41:11.123 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:41:11.127 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:41:11.127 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-09-19T07:41:11.127 INFO:teuthology.orchestra.run.smithi189.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +795us[ +773us] +/- 47ms 2024-09-19T07:41:11.127 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 8 -2562us[-2584us] +/- 86ms 2024-09-19T07:41:11.127 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +749us[ +749us] +/- 72ms 2024-09-19T07:41:11.127 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:41:11.128 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:41:11.129 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-09-19T07:41:11.129 INFO:teuthology.orchestra.run.smithi203.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 +776us[ +791us] +/- 47ms 2024-09-19T07:41:11.129 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 -2584us[-2569us] +/- 86ms 2024-09-19T07:41:11.129 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 +713us[ +728us] +/- 72ms 2024-09-19T07:41:11.129 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:41:11.138 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:41:11.145 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:41:11.146 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-09-19T07:41:11.146 INFO:teuthology.orchestra.run.smithi159.stdout:^* hv01.front.sepia.ceph.com 3 7 377 5 +873us[ +868us] +/- 47ms 2024-09-19T07:41:11.146 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 -2505us[-2509us] +/- 86ms 2024-09-19T07:41:11.146 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 +770us[ +765us] +/- 72ms 2024-09-19T07:41:11.146 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:41:11.147 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T07:41:11.156 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T07:41:11.156 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T07:41:11.176 DEBUG:teuthology.orchestra.run.smithi159:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T07:41:11.315 DEBUG:teuthology.orchestra.run.smithi189:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T07:41:11.384 DEBUG:teuthology.orchestra.run.smithi203:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T07:41:11.453 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:41:11.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:41:11.514 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:41:11.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:41:11.543 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:41:11.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:41:11.571 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T07:41:11.581 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T07:41:11.618 INFO:teuthology.task.internal:Duration was 1581.924290 seconds 2024-09-19T07:41:11.619 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T07:41:11.653 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T07:41:11.653 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T07:41:11.658 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T07:41:11.660 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T07:41:11.699 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:41:11.699 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:41:11.740 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:41:12.193 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T07:41:12.194 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi159.front.sepia.ceph.com 2024-09-19T07:41:12.194 DEBUG:teuthology.orchestra.run.smithi159:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T07:41:12.244 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi189.front.sepia.ceph.com 2024-09-19T07:41:12.244 DEBUG:teuthology.orchestra.run.smithi189:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T07:41:12.273 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi203.front.sepia.ceph.com 2024-09-19T07:41:12.273 DEBUG:teuthology.orchestra.run.smithi203:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T07:41:12.296 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T07:41:12.296 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T07:41:12.300 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T07:41:12.315 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T07:41:12.362 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T07:41:12.362 DEBUG:teuthology.orchestra.run.smithi159:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T07:41:12.367 DEBUG:teuthology.orchestra.run.smithi189:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T07:41:12.383 DEBUG:teuthology.orchestra.run.smithi203:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T07:41:12.918 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T07:41:12.930 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T07:41:12.930 DEBUG:teuthology.orchestra.run.smithi159:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T07:41:12.986 DEBUG:teuthology.orchestra.run.smithi189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T07:41:13.013 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T07:41:13.039 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T07:41:13.049 DEBUG:teuthology.orchestra.run.smithi159:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:41:13.054 DEBUG:teuthology.orchestra.run.smithi189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:41:13.056 DEBUG:teuthology.orchestra.run.smithi203:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:41:13.078 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = core 2024-09-19T07:41:13.103 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = core 2024-09-19T07:41:13.120 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = core 2024-09-19T07:41:13.150 DEBUG:teuthology.orchestra.run.smithi159:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:41:13.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:41:13.184 DEBUG:teuthology.orchestra.run.smithi189:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:41:13.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:41:13.199 DEBUG:teuthology.orchestra.run.smithi203:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:41:13.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:41:13.214 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T07:41:13.225 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T07:41:13.225 DEBUG:teuthology.misc:Transferring archived files from smithi159:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911794/remote/smithi159 2024-09-19T07:41:13.226 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T07:41:13.301 DEBUG:teuthology.misc:Transferring archived files from smithi189:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911794/remote/smithi189 2024-09-19T07:41:13.301 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T07:41:13.338 DEBUG:teuthology.misc:Transferring archived files from smithi203:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911794/remote/smithi203 2024-09-19T07:41:13.338 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T07:41:13.377 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T07:41:13.377 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T07:41:13.381 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T07:41:13.383 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T07:41:13.425 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T07:41:13.438 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T07:41:13.438 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T07:41:13.461 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T07:41:13.461 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T07:41:13.466 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T07:41:13.468 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T07:41:13.481 INFO:teuthology.orchestra.run.smithi189.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 07:41 /home/ubuntu/cephtest 2024-09-19T07:41:13.483 INFO:teuthology.orchestra.run.smithi203.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 07:41 /home/ubuntu/cephtest 2024-09-19T07:41:13.496 INFO:teuthology.orchestra.run.smithi159.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 07:41 /home/ubuntu/cephtest 2024-09-19T07:41:13.498 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T07:41:13.509 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T07:41:13.564 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/iozone}} duration: 1581.9242897033691 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T07:41:13.564 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T07:41:13.651 INFO:teuthology.run:pass